topic Re: confidence function in Mathcad vs. manual calculation in Mathcad

confidence function in Mathcad vs. manual calculation

WalterSchrabmai — Thu, 03 May 2018 14:25:36 GMT

Hi friends,I really can not find the error, why the manual calculation of the std errors of the parameters are completely wrong compared with confidence function.Can anyone help, please?Walter

Re: confidence function in Mathcad vs. manual calculation

WalterSchrabmai — Fri, 03 Jun 2016 18:49:20 GMT

I found it: Ks and Kp was changed in the Model definition. and of course I must fit the function before. sorry

Re: confidence function in Mathcad vs. manual calculation

WalterSchrabmai — Fri, 03 Jun 2016 19:14:40 GMT

No!! I can not find the error in my sheet. Please help.

Here confidence.

Re: confidence function in Mathcad vs. manual calculation

RichardJ — Fri, 03 Jun 2016 20:00:57 GMT

Your Covar matrix is wrong. The elements on the diagonals are variances. Since the variance is the square of the standard deviation it should never be negative.

Re: confidence function in Mathcad vs. manual calculation

WalterSchrabmai — Fri, 03 Jun 2016 20:12:19 GMT

Thanks, but the Covar calculation might be correct. Why is the Jacobian not correct? Could you take a minute to check the sheet, please?

Re: confidence function in Mathcad vs. manual calculation

WalterSchrabmai — Fri, 03 Jun 2016 20:35:18 GMT

When I do a symbolic diff (made in Mathematica and inserted as formula in the sheet on the beginning as region) the results are the same.

Re: confidence function in Mathcad vs. manual calculation

RichardJ — Fri, 03 Jun 2016 20:58:40 GMT

Right now, I can't see the problem either. The math looks correct, the implementation looks correct, and yet Covar is clearly wrong.

Re: confidence function in Mathcad vs. manual calculation

WalterSchrabmai — Fri, 03 Jun 2016 21:01:05 GMT

The strange thing is, when I put the data for curve 5 in my big sheet, it works fine. Even when I just calculate the 5th curve (below in the sheet)

I have just drag and paste the part in my short sheet.

Re: confidence function in Mathcad vs. manual calculation

RichardJ — Fri, 03 Jun 2016 21:13:18 GMT

I think it's a numerical problem. I displayed A at full precision (17 digits), copied the contents to a new sheet, and the numeric engine can't invert the matrix. If you check the determinant of AT*A is very small:

Re: confidence function in Mathcad vs. manual calculation

WalterSchrabmai — Fri, 03 Jun 2016 21:31:31 GMT

Thanks, yes but why it works in the big sheet where I have 5 curves with different S0 but the same V, Ks, Kp. But you could be right. I will check the Matrixinversion in MMA tomorrow. Thanks a lot for your help.

Re: confidence function in Mathcad vs. manual calculation

WalterSchrabmai — Sat, 04 Jun 2016 06:27:12 GMT

Dear friends,

I have done it also with Mathematica and got an INVERSE Error. BUT HOW can Mathematica then calculate the stderr for the Parameter with the build-in package? SO the same is with MathCAD too.

Re: confidence function in Mathcad vs. manual calculation

RichardJ — Sat, 04 Jun 2016 14:25:51 GMT

They obviously use algorithms that are more stable for near singular matrices. Fortunately, Mathcad does offer such an algorithm with the function geninv:

Re: confidence function in Mathcad vs. manual calculation

WalterSchrabmai — Sat, 04 Jun 2016 16:39:10 GMT

Thanks you Richard, but it depends also on the simulation data I think. Because in this model I still have great differeneces.

Maybe you could take a lot. I have there 2 cases for the model. one with 2paramter- Quadratic model and the othere the MichaelisMenten model.

The Quadratic works fine, but the MM not alsways. I would be nosey for your MC sheet where you have this cute stderr for the parameter from your picture.

Anyway thanks a lot.

Re: confidence function in Mathcad vs. manual calculation

RichardJ — Sat, 04 Jun 2016 20:52:18 GMT

Thanks you Richard, but it depends also on the simulation data I think. Because in this model I still have great differeneces.

Look at the determinant of AT*A. Now it's even smaller: about 10^-29. So just before the call to geninv put TOL:=10^-6. Then the numbers are closer, although still different. Change TOL to 10^-7. The manually calculated numbers don't change this time. That doesn't necessarily mean the manually calculated numbers are wrong though, it means that either the manually calculated numbers are wrong, or the ones from the built-in function are wrong, or both. However, since the geninv algorithm is iterative (it has to be, otherwise the results would not depend on TOL), and appears to have converged to a stable solution with TOL set to 10^-6 or less, I'm inclined to believe the manual calculation is now correct, and the built-in function is wrong because of numerical problems.

I would be nosey for your MC sheet where you have this cute stderr for the parameter from your picture.

It's just your sheet with the calculation of the inverse changed to use geninv. There are no other differences.

Re: confidence function in Mathcad vs. manual calculation

WalterSchrabmai — Sat, 04 Jun 2016 21:39:34 GMT

Thank you, Richard for your great explanation! Now I have understood.

Re: confidence function in Mathcad vs. manual calculation

AlanStevens — Sun, 05 Jun 2016 07:29:57 GMT

Along similar lines: The rank of your 3x3 matrix, ATA, is only 2. If you look at the reduced row echelon form, rref(ATA), you will see that the bottom row is all zeros.

Alan

Re: confidence function in Mathcad vs. manual calculation

WalterSchrabmai — Sun, 05 Jun 2016 10:44:37 GMT

Thanks, I see. what does rref mean, and how I can interpret the result from rref? I think that the rank(ATA)= always 2 because also in this sheet, where it works, as the solution from Richards is used. Please see.

Re: confidence function in Mathcad vs. manual calculation

RichardJ — Sun, 05 Jun 2016 13:51:12 GMT

the bottom row is all zeros.

Close to all zeros, but not all zeros:

Re: confidence function in Mathcad vs. manual calculation

AlanStevens — Sun, 05 Jun 2016 17:37:38 GMT

Richard Jackson wrote:

Close to all zeros, but not all zeros:

I admire your dedication to exactitude Richard! However, this is just numerical imprecision. In reduced row echelon form, this is going to be either a 1 or a zero. I suggest that 3.224x10^-20 is closer to zero!

Alan

Re: confidence function in Mathcad vs. manual calculation

WalterSchrabmai — Sun, 05 Jun 2016 17:37:48 GMT

OK, here is my final resut of the sheet. Now it works fine, with noise adding to great test data. Thanks to all of you!