James Rogers <jamesr@best.com>, Mon Oct 15, 2001 :
> On 10/14/01 10:37 PM, "Amara Graps" <Amara.Graps@mpi-hd.mpg.de> wrote:
>> (It's not published yet.) The idea is to use the statistical
>> characteristics of the data to fill in gaps for modelling
>> purposes. Note: you are NOT creating *real* data (i.e. the folks who
>> need to make predictions based on real data should not use this). But
>> I think that this method has a valid use for those folks who can't
>> apply a data analysis on their data because of small gaps in the
>> time-series, or are trying to show long term dynamics of a system.
>> For example, the 'standard' wavelet transforms which determine
>> frequencies require as input time series data on an evenly-spaced time
>> grid.
>Just out of curiosity, why couldn't full spectral re-synthesis be
>used to interpolate the data rather than using statistical models?
I don't know the answer, but I'll venture some guesses, assuming
that by "spectral" you mean Fourier decomposition and
recomposition methods. Gaps in the data would produce fictitious
peaks at low frequencies, where the wavelengths are comparable to
the gaps. Also, I think that you have to be careful, when using
Fourier methods, that the data is stationary, and my own experience
in scientific time series is that stationarity occurs rarely.
>I tend
>to shy away from statistical models because many, though not all,
>statistical models seem to be sensitive to anomalies in the data
>i.e. even many of the adaptive ones are based on the "expected case"
>and can do ugly things when they come across something unusual.
I think that there is a trend to use wavelets and beyond for those
time-series that have anomalies. By "beyond" I mean multiple
wavelets, wavelet packets, cosine packets, chirplets, warplets.
Mallat, in _A Wavelet Tour of Signal Processing_ says that creating
new basis families may become just a popular new sport of basis
hunting if not motivated by application.
>Unrelated to this discussion, it kind of shocks me to see some of
>the archaic methods of analyzing and working with time series data
>used in many parts of industry and even science and engineering. The
>engineering discipline of signal processing has very mature and
>extremely generalized mathematics for handling just about any aspect
>of generic time series data you could want in many cases, but many
>of the algorithms are rarely applied outside of that discipline.
OK you're shocked, but the reality is that scientists don't have
enough hours in the day to be up-to-speed on the latest techniques.
In the astronomy that I know about, helioseismologists are the most
aware about digital signal processing because the Sun acts like
a ringing gas bell providing one million oscillation modes that
need to be sorted out. The seismic geophysicists know a bit more
because the Earth's interior presents a sticky inhomogeneous challenge
to find the oscillations. Helioseismolgists frequently go to
geophysics meetings to be exposed to the sophisticated techniques
that the other field uses. The digital signal processing people
have the most sophisticated techniques, but then you have to allocate
alot of time to discover them. How to do that?
Scientists could choose to go to large meetings where only 1% of
people in their field are present, in order to be exposed to more
fields and problem-solving methods, but then they risk that it might
be a lot of money and precious time spent on the possibility that
they will not get their hands on that special method that will solve
their scientific problem better. Or they could choose to spend their
precious time and money only on the science meetings where they will
have the maximum interacton with people in their own field. My
meeting-attendence pattern during the last few years was more like
the latter. I only encountered the paper above because I scheduled
part of my holiday last month to spend with a group of volcanologists.
The following link points to some of the mathematical methods
on which a scientist should be up-to-speed (in my opinion),
and I think it's an impossible task, given their time constraints:
http://www.amara.com/science/science.html#num
Here's a money-making hint. Scientists need the help of scientific
programmers and numerical-methods specialists who are knowledgeable
about the latest and best numerical methods. (I would dearly love to
hire AmaraGrapsv.1991, because she was much more aware of these
methods than AmaraGrapsv.2001, and I hear her rates were cheap.) If
you can find a way to fill this need given the paperwork and
bureaucratic nonsense and garbage when dealing with
government-funded science, please try. Ten years ago I wrote a
proposal for setting up a "Scientific Computing Center" at NASA-Ames
to exactly do this, and it wasn't funded. Maybe I didn't present it
right, or the time wasn't right, but the need still exists.
Amara
P.S. Another paper by the same author as that discussed above is:
"Stochastic modelling at Stromboli: a volcano with remarkable
memory" by O. Jaquet and R. Carniel, Journal of volcanolgy and
Geothermal Research 105 (2001) 249-262.
********************************************************************
Amara Graps, PhD email: amara@amara.com
Computational Physics vita: ftp://ftp.amara.com/pub/resume.txt
Multiplex Answers URL: http://www.amara.com/
********************************************************************
"Take time to consider. The smallest point may be the most essential."
Sherlock Holmes (The Adventure of the Red Circle)
This archive was generated by hypermail 2b30 : Sat May 11 2002 - 17:44:16 MDT