Integrating data

[Jbryant61]

I often have dye spectra that I need to integrate. I usually do this by spline-ing the data and then integrating the spline.

My colleague ha shown me how in matlab there is a simple command called sum that just summs all the data.

Is this possible in mathcad?

thanks
Jason

[StuartBruff]

On 3/5/2009 6:31:50 AM, Jbryant61 wrote:
I often have dye spectra that I need to integrate. I usually do this by spline-ing the data and then integrating the spline.
My colleague ha shown me how in matlab there is a simple command called sum that just summs all the data.

Do you mean this command?

Is this possible in mathcad?

If so, then yes. Just use the vector sum operator, ctl-4 or from the Vector and Matrix Operators toolbar.

Stuart

[Jbryant61]

But you can't specify a "windowed" region to perform the sum,.

thanks
jason

[StuartBruff]

On 3/5/2009 7:36:34 AM, Jbryant61 wrote:
== But you can't specify a "windowed" region to perform the sum,.

You didn't ask for a "windowed" summation; you asked for a command that sums all of the data?

Depending what you mean by "windowed", you could use either the summation operator (ctl-shift-4) or range sum operator ($); the former allows you to give explicit lower and upper bounds, the latter does what it says on the tin.

Stuart

[PhilipOakley]

Just the vector sum that allows you to select which range of indexes to uses (shortcut ctrl+$)

If you want to weight the data, just stick that as part of the formula as us sum over the paired product of the vector and the weight.

As a side note, the integral of the spline is essentially the same as the Simpson's rule summation. It takes a few moments to manipulate the formulae to show the pairings and the minor residual difference, but essentially they are the same. They use the same interpoint formulation assumptions.

second side note: in terms of degrees of freedom and variance of the weighted sum, look up the Welsh-Satterthwaite formula (as used in the GUM - Guide to Uncertainty in Measurement)

Philip Oakley

[TomGutman]

>>As a side note, the integral of the spline is essentially the same as the Simpson's rule summation. It takes a few moments to manipulate the formulae to show the pairings and the minor residual difference, but essentially they are the same. They use the same interpoint formulation assumptions.<<

That is not true. Simpsons rule is equivalent to fitting a cubic through each triple of points, with the requirement that the middle point be at the center. The spline fits a different cubic between each pair of points. Not the same thing at all, even when Simpson's rule can be applied.

As an additional side note, once the cubic spline is calculated there is no need for the additional error from a numeric integration. One can easily calculate the exact integral of the cubic spline, functions to do that are included in my cubic spline worksheet.
__________________
� � � � Tom Gutman

[ptc-1368288]

>As a side note, the integral of the spline is essentially the same as the Simpson's rule summation <.
__________________

If you takes the spline points to integrate as the X, Y set. But then there is no need for splines. Splines are essentially interpolation methods and can't be any better than Simpson or other applicable quadratures [Riemann, Poncelet ...]. Splines are scalar and behave like functions but don't recreate a true function.

What I meant is that what you said is not true.

jmG

[PhilipOakley]

>>As a side note, the integral of the spline is essentially the same as the Simpson's rule summation <.

The key word was essentially

That is, the answers are not exactly the same but, with reasonable caveats, they are pretty close.

The main point I was trying to note is that it isn't necessary to do the spline and integral (with fine precision). One is gaining a false sense of accuracy from doing the spline and then its integral. There is no more data. Only a choice of approximations.

The comment is for the general reader. Tom, Jean & I have discussed this in previous threads in different forms. It is worth the reader digging out the formulas so they can make reasoned choices for their own special data 😉 [Choose your caveats!]

Philip Oakley

[TomGutman]

You are correct in that there is no more data, just a choice of approximations. But then, if any approximation is as good as any other approximation, why bother with even Simpson's rule? Why not just a simple Euler sum? The cubic spline uses more degrees of freedom (a cubic for each interval instead of a cubic for each pair of intervals) to produce a smoother spline (C² rather than C¹), and so can be expected to be a closer approximation to an underlying analytic (C^∞) function. How significant this improvement is in any particular case is open to question. But then, how much harder is it to do the cubic spline than Simpson's rule (using available toolkits)? Why not use the best easily available procedure?
__________________
� � � � Tom Gutman

[TomGutman]

Here is a sheet comparing different integrations for a data set. Simpson's rule, where applicable, is indeed fairly close to the cubic spline method. But by no means equivalent. Users will have to judge for their own data whether the extra error with Simpson's rule justifies the extra coding for Simpson's rule.
__________________
� � � � Tom Gutman

[Jbryant61]

Hi, thanks everyone for your comments.

For my application (integrating spectra using tabulated data with data points every 0.5nm or so), I am happy with the spline (I currently use about 2 lines of coding). I was only asking about alternative "built in" methods if they were simpler to implement (albeit with some error). In Matlab, there was a one line sum command where you specify the data and the x values for the windowed part). It seems there isn't a simple command in Mathcad, but one could be written e.g by using the submatrix command.



thanks
Jason

[ptc-1368288]

I disagree, but that's it.

The finite differences is the Engineer yard stick. You can window submatrix and there is no coding as such as it is part of the utilities like so many that make Mathcad so powerful, flexible and traceable. Splinning a spectral data set is not to be argued a profane escape.

Unless modified, Simpson is not cumulative, finite differences is cumulative, independent of the spacing and that's what is most wanted, i.e: see the evolution in the spectral domain or if you wish in the Fourier coefficients domain.

A work sheet is always the key to project solution.

jmG

[StuartBruff]

On 3/6/2009 3:28:26 AM, Jbryant61 wrote:
== In Matlab, there was a one line sum command where you specify the data and the x values for the windowed part)

What is the name of that 'sum' command? It's not the standard function 'sum'.

Stuart

[TomGutman]

The four place holder Σ will easily sum any part of a vector. Although the limits are specified in terms of the indices, not x values. But in any case, a simple sum is a very poor estimate of an integral. Why use it? It's ths sort of thing people without real tools, using something like Excel, may have to live with, but not Mathcad users.
__________________
� � � � Tom Gutman

[PhilipOakley]

On 3/5/2009 9:08:26 PM, Tom_Gutman wrote:
>Here is a sheet comparing
>different integrations for a
>data set. Simpson's rule,
>where applicable, is indeed
>fairly close to the cubic
>spline method. But by no
>means equivalent. Users will
>have to judge for their own
>data whether the extra error
>with Simpson's rule justifies
>the extra coding for Simpson's
>rule.
__________________
� � �
>� Tom Gutman

A useful comparison. The "previous thread" was where someone had asked to 'integrate a vector' and we discussed the comparison of methods at that point as well. It is probably still a need for uses to be able to simply 'integrate a vector' (in one line ;-).


Philip Oakley

[ptc-1368288]

On 3/6/2009 4:19:56 AM, philipoakley wrote:
...
>... It is probably still a need for
>uses to be able to simply 'integrate a
>vector' (in one line ;-).
>
>Philip Oakley
_________________________

The finites differences is one (1) line.
The bonus is the cumulative, as demonstrated.
If all the modules exchanged in this collab
would be "built-in" the list will be several 1000's.

Not bad an idea providing a user coding
simple ... bingo in the function list .

No good for exchanging work sheets with missing links [fnct].

jmG

[TomGutman]

You can't just integrate a vector. You have to have the ordinate values. They provide a scale, at the very least.

What means "one line"? The sheet I've posted has four integration functions, any of them can be evaluated in one line. It's just a question of what you consider to be your tools. Note that much of what is often considered to be a part of Mathematica is not really. It is in a variety of packages, mostly written in Mathematica itself, and which must be invoked to make their components available. Perhaps a way of better hiding what is done, but Mathcad has referenced worksheets and collapsed areas to serve the same purpose. I see little difference between a Mathcad built in function and one provided for in a more or less standard toolbox.

Yes, the topic has come up before. And the statement that Simpson's rule and the cubic spline are the same was made there, based on the erroneous assumption that since Simpson's rule is exact on a cubic, it will be exact on the cubic spline. But a cubic spline is not a cubic, and while Simpson's rule could be used to get an exact integral for the spline to do so requires using additional evaluation points, points halfway between the given data. Applied to the original data it is not the same as the cubic spline.

I also see no reason to seriously consider it. It is more restrictive (requiring an odd number of points, and, in most implementations, requiring uniform sampling, at the very least requiring pairs of equal intervals), less accurate, requires more code. What's to like about it? I could make a better case for the trapezoidal rule for some data.
__________________
� � � � Tom Gutman

[ptc-1368288]

On 3/5/2009 3:13:24 PM, philipoakley wrote:
...
>The comment is for the general
>reader. Tom, Jean & I have
>discussed this in previous
>threads in different forms. It
>is worth the reader digging
>out the formulas so they can
>make reasoned choices for
>their own special data 😉
>[Choose your caveats!]
>
>Philip Oakley
______________________________

Numerical integration have been exhausted c/w Mathcad work sheets. The points in the attached are pretty obvious:
1. A well behaved function should go Romberg or other built-in. It will gladly fails unsuspected if not careful.
2. The finite differences remains the "yardstick" for any case of data sets: noisy, unequally spaced points.
3. spline has no interest in general and should in fact not be considered a priori. Fitting/smoothing should be the way to go.
4. The "College Quadratures" have limited use in applied maths. Each of the 3 methods is exact for certain types of functions. Only on known cases the choice will dictate.
4.0. Riemann= rectangles
4.1. Poncelet = Trapezes
4.2. Simpson = cubic

5. Advanced methods
5.0. Legendre
5.1. Hermite
5.2. Laguerre
5.3. Radau
5.4. Chebyshev
5.5. Lobatto
5.6. Jacobi
5.7. Simplex: simple, double, triple

6. Points Integration for "on the fly" integration.

All methods and Mathcad work sheets have been published in the forum.
Always cross check serious projects.

Version 8 attached for a large audience.

jmG

[ptc-1368288]

On 3/5/2009 7:36:34 AM, Jbryant61 wrote:
>But you can't specify a
>"windowed" region to perform
>the sum,.
>
>thanks
>jason
_____________________________

Sure you can by submatrix.

Spline is not needed and by no means more accurate, use the finite differences as exemplified in the attached. The vector sum gives the terminal value. The finite differences gives the running "cumulative Integral", hyperfast, exact, supports non equally spaced points (within reason). Window as desired by submatrix.

jmG