Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coefficients and FIR filters must have at least one numerator #127

Open
crotwell opened this issue Oct 1, 2020 · 14 comments
Open

Coefficients and FIR filters must have at least one numerator #127

crotwell opened this issue Oct 1, 2020 · 14 comments

Comments

@crotwell
Copy link
Contributor

crotwell commented Oct 1, 2020

Mathematically, a Coefficients filter has to have at least one numerator, which can be 1. Otherwise the output time series would be all zeros.

A no-coefficient Coefficient filter might be used to just be a container for units, but this should be discouraged. Requiring at least one numerator would force using of at least a single coefficient, even if it is 1.

Similarly a FIR filter should have at least one NumeratorCoefficient

Related issue to change stationxml schema to enforce this here

@timronan
Copy link
Collaborator

timronan commented Oct 1, 2020

Is this a stationxml-validator rule or a schema rule? We should not make redundant validation checks. If this is changed in the schema it would become part of the data definition rather than a rule that is only enforced on metadata stored in the IRIS database.

@crotwell
Copy link
Contributor Author

crotwell commented Oct 1, 2020

I would prefer it eventually be a schema rule, but I suspect that process will take a looooooong time as it would have to be a major version as is a "breaking change".

Validator rule for now, maybe a schema rule if/when stationxml 2.0 comes out.

@timronan
Copy link
Collaborator

timronan commented Oct 2, 2020

I just built a resp file from a dataless file, which was converted from a stationxml, that contains a Coefficient with 0 numerators and 0 denominators. It is attached. This resp file returns a non-zero response when it is processed using evalresp. Evalresp may have a built in way to handle this edge case. There is nothing in the SEED manual that states you can or cannot have a Coefficient with 0 numerators.

I wonder if there are any other dependent programs that could be affect by Coefficients with zero numerators.

Look at stage 3.

RESP.NUMERATOR.TEST.zip

@metempleton
Copy link
Collaborator

metempleton commented Oct 2, 2020 via email

@crotwell
Copy link
Contributor Author

crotwell commented Oct 2, 2020

Humm, I think I would agree with Mary that this is evalresp handling an edge case. Mathematically wrong, but probably the right thing for software to do. A test would be to create the same stationxml, except with a single numerator with value 1 and see if evalresp outputs the same response. Hopefully it does.

I guess it could be that this is something that has been done wrongly for so long that it is hard to fight against, but it is wrong. So I don't know...

FWIW, it filed an issue on the stationxml schema to require at least one numerator here:
FDSN/StationXML#56
but of course that will be a long time in being approved, if ever.

@timronan
Copy link
Collaborator

timronan commented Oct 2, 2020

Is the mathematical function being described in this issue equation 8 on pg 161 of the SEED manual? I'll run the elavresp test. I want to make sure I am looking at the correct equation that describes a coefficient. Any reference outlining the equation will work. Thanks

@metempleton
Copy link
Collaborator

metempleton commented Oct 2, 2020 via email

@crotwell
Copy link
Contributor Author

crotwell commented Oct 2, 2020

Actually kind of sort of like with L = -1. Which is impossible. L=0 implies L+1=1 coefficients, and that at least b_0 exists.

This is why I am saying it is mathematically wrong. If there are no coefficients, then there isn't even a b_0. Or perhaps another way of thinking of it is the sum is really over infinitely many terms,
y_k = b_0 x_k-0 + b_1 x_k-1 + b_2 x_k-2 + ...
where to avoid writing, we skip all the terms where b_n is zero. But if you have no coefficients, or they are all zero, your equation is:
y_k = ?

I would say that if there are no coefficients, then this means y_k = 0 since the sum is over only zero terms, but you could just as easily say it is not defined at all.

On the other hand, a coefficients filter that does nothing, just passes the data on without change, would be
y_k = x_k
or
y_k = b_0 * x_k-0 where b_0 is 1. So a single numerator with value 1.

These degenerate cases are tricky, which is why I would prefer they aren't used in actual metadata.

Also, sorry for linking the schema issue twice. I have too many issues rolling around in my head lately and can't keep them straight! :(

@timronan
Copy link
Collaborator

timronan commented Oct 2, 2020

Okay, thanks for the clarification. Yeah, this is a mathematically wrong and should probably be fixed in the metadata even though evalresp automatically handles the case.

After checking, evalresp seems to adds a unity numerator to coefficients where numerators are zero. Evalresp outputs the same result for a resp file with no numerator and for one where the numerator is 1. A coefficient with a zero numerator should be an error. Just because evalresp is smart enough to catch this case does necessarily mean other response processing programs will be.

So the rules would be:

408 | IF Stage[N]:Coefficient is included than 1 or more NonZero Stage[N]:Coefficient:Numerator must be included | Error
409 | IF Stage[N]:FIR is included than 1 or more NonZero Stage[N]:FIR:Numerator must be included | Error

I want to add a NonZero clause into this rule but I don't think it completely correct. It seems mathematically correct to have a zero numerator, I am not sure why it would be used, if there is more than one numerators. Are there any thoughts about whether it is acceptable to have a zero numerator?

Maybe the rules could be combined into one rule.

Both rules are errors and have no exceptions. Don't worry about the double link.

@crotwell
Copy link
Contributor Author

crotwell commented Oct 2, 2020

I think you are right, it is ok mathematically to have numerator coefficients that are all zeros, but it is pointless as a response, as it says the output is always zero. If the output is always zero regardless of the input, why bother recording it? :)

So +1 on the NonZero

@metempleton
Copy link
Collaborator

metempleton commented Oct 2, 2020 via email

@crotwell
Copy link
Contributor Author

crotwell commented Oct 2, 2020

So the denominators are coefficients of previous output values, but without at least a single non-zero numerator, there is no way for any previous output values to ever become non-zero, so it would still always be zero.

An non-zero IIR filter still needs at least one non-zero numerator to generate a non-zero output.

@metempleton
Copy link
Collaborator

metempleton commented Oct 5, 2020 via email

@timronan
Copy link
Collaborator

timronan commented Oct 5, 2020

Thanks for the clarification. So the purposed rules will read as:

408 | IF Stage[N]:Coefficient is included than 1 or more NonZero Stage[N]:Coefficient:Numerator must be included | Error
409 | IF Stage[N]:FIR is included than 1 or more NonZero Stage[N]:FIR:Numerator must be included | Error

Maybe these rule will encourage the numerator schema decision to be re-evaluated. Thank you both very much for helping explain this rule.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants