**Next message:**Brian Phillips: "A renaissance for civil defense?"**Previous message:**Robert J. Bradbury: "Re: Private vs. government views of property"**Maybe in reply to:**James Rogers: "Data reconstruction (was Dithering)"**Next in thread:**J. R. Molloy: "Re: Data reconstruction (was Dithering)"**Reply:**J. R. Molloy: "Re: Data reconstruction (was Dithering)"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ]

James Rogers <jamesr@best.com>, Mon Oct 15, 2001 :

*> On 10/14/01 10:37 PM, "Amara Graps" <Amara.Graps@mpi-hd.mpg.de> wrote:
*

*>> (It's not published yet.) The idea is to use the statistical
*

*>> characteristics of the data to fill in gaps for modelling
*

*>> purposes. Note: you are NOT creating *real* data (i.e. the folks who
*

*>> need to make predictions based on real data should not use this). But
*

*>> I think that this method has a valid use for those folks who can't
*

*>> apply a data analysis on their data because of small gaps in the
*

*>> time-series, or are trying to show long term dynamics of a system.
*

*>> For example, the 'standard' wavelet transforms which determine
*

*>> frequencies require as input time series data on an evenly-spaced time
*

*>> grid.
*

*>Just out of curiosity, why couldn't full spectral re-synthesis be
*

*>used to interpolate the data rather than using statistical models?
*

I don't know the answer, but I'll venture some guesses, assuming

that by "spectral" you mean Fourier decomposition and

recomposition methods. Gaps in the data would produce fictitious

peaks at low frequencies, where the wavelengths are comparable to

the gaps. Also, I think that you have to be careful, when using

Fourier methods, that the data is stationary, and my own experience

in scientific time series is that stationarity occurs rarely.

*>I tend
*

*>to shy away from statistical models because many, though not all,
*

*>statistical models seem to be sensitive to anomalies in the data
*

*>i.e. even many of the adaptive ones are based on the "expected case"
*

*>and can do ugly things when they come across something unusual.
*

I think that there is a trend to use wavelets and beyond for those

time-series that have anomalies. By "beyond" I mean multiple

wavelets, wavelet packets, cosine packets, chirplets, warplets.

Mallat, in _A Wavelet Tour of Signal Processing_ says that creating

new basis families may become just a popular new sport of basis

hunting if not motivated by application.

*>Unrelated to this discussion, it kind of shocks me to see some of
*

*>the archaic methods of analyzing and working with time series data
*

*>used in many parts of industry and even science and engineering. The
*

*>engineering discipline of signal processing has very mature and
*

*>extremely generalized mathematics for handling just about any aspect
*

*>of generic time series data you could want in many cases, but many
*

*>of the algorithms are rarely applied outside of that discipline.
*

OK you're shocked, but the reality is that scientists don't have

enough hours in the day to be up-to-speed on the latest techniques.

In the astronomy that I know about, helioseismologists are the most

aware about digital signal processing because the Sun acts like

a ringing gas bell providing one million oscillation modes that

need to be sorted out. The seismic geophysicists know a bit more

because the Earth's interior presents a sticky inhomogeneous challenge

to find the oscillations. Helioseismolgists frequently go to

geophysics meetings to be exposed to the sophisticated techniques

that the other field uses. The digital signal processing people

have the most sophisticated techniques, but then you have to allocate

alot of time to discover them. How to do that?

Scientists could choose to go to large meetings where only 1% of

people in their field are present, in order to be exposed to more

fields and problem-solving methods, but then they risk that it might

be a lot of money and precious time spent on the possibility that

they will not get their hands on that special method that will solve

their scientific problem better. Or they could choose to spend their

precious time and money only on the science meetings where they will

have the maximum interacton with people in their own field. My

meeting-attendence pattern during the last few years was more like

the latter. I only encountered the paper above because I scheduled

part of my holiday last month to spend with a group of volcanologists.

The following link points to some of the mathematical methods

on which a scientist should be up-to-speed (in my opinion),

and I think it's an impossible task, given their time constraints:

http://www.amara.com/science/science.html#num

Here's a money-making hint. Scientists need the help of scientific

programmers and numerical-methods specialists who are knowledgeable

about the latest and best numerical methods. (I would dearly love to

hire AmaraGrapsv.1991, because she was much more aware of these

methods than AmaraGrapsv.2001, and I hear her rates were cheap.) If

you can find a way to fill this need given the paperwork and

bureaucratic nonsense and garbage when dealing with

government-funded science, please try. Ten years ago I wrote a

proposal for setting up a "Scientific Computing Center" at NASA-Ames

to exactly do this, and it wasn't funded. Maybe I didn't present it

right, or the time wasn't right, but the need still exists.

Amara

P.S. Another paper by the same author as that discussed above is:

"Stochastic modelling at Stromboli: a volcano with remarkable

memory" by O. Jaquet and R. Carniel, Journal of volcanolgy and

Geothermal Research 105 (2001) 249-262.

********************************************************************

Amara Graps, PhD email: amara@amara.com

Computational Physics vita: ftp://ftp.amara.com/pub/resume.txt

Multiplex Answers URL: http://www.amara.com/

********************************************************************

"Take time to consider. The smallest point may be the most essential."

Sherlock Holmes (The Adventure of the Red Circle)

**Next message:**Brian Phillips: "A renaissance for civil defense?"**Previous message:**Robert J. Bradbury: "Re: Private vs. government views of property"**Maybe in reply to:**James Rogers: "Data reconstruction (was Dithering)"**Next in thread:**J. R. Molloy: "Re: Data reconstruction (was Dithering)"**Reply:**J. R. Molloy: "Re: Data reconstruction (was Dithering)"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ]

*
This archive was generated by hypermail 2b30
: Sat May 11 2002 - 17:44:16 MDT
*