Usage of pyextremes for non time series data

[Dillion]

Is there any way to use pyextremes threshold selection for non time series data? I'm interested in modelling the tail distribution for a series of values with no datetime or timedelta, it seems that mean residual life plot works without the datetime index but not the rest of the plots in the threshold selection examples. What parameters should I pass in for the other plots or how should I modify my data series if I want to apply the analysis to the whole series and time granularity doesn't matter?

[georgebv]

What is the question you are trying to answer by performing this analysis?

For example, extreme value analysis (pyextremes is a framework for univariate EVA) answers the following question:

What is the value that has a given exceedance probability within a given time period?

Or, the inverse of the above:

What is the probability that a given value is exceeded within a given time period?

And here are some practical examples:

• You live near ocean and need to find a wave that has a 1% probability of being exceeded in any given year. You need this because you need to choose how high your house needs to be built above the ground. You perform an analysis and the answer is, say, 6 meters.
• You have a wall that's 3 meters and you need to know how often waves are going to wash over it. You run the analysis and find out that, say, there is a 20% probability in any given year that it will be overtopped.

[Dillion]

I would like to find the threshold at which the occurrences of values above this threshold significantly increase, so that I can use this threshold as a cutoff for classifying extreme events. Looking at some statistical tests of my data for example Anderson-Darling test, it seems that the distribution deviates from normal distribution especially in the right tail, and I want to understand where in the right tail results in a stable number of extreme events.
I can manually set different thresholds to find acceptable cutoffs, but from what I understand GPD (if it is an appropriate model to use for my distribution) can provide a more rigorous method of finding the threshold by looking at the stability of the excesses over threshold. Regarding the 'given time period', it can be considered that the current values that I have is a snapshot of the data over the whole time period I am interested in, so maybe I can just define an arbitrary time scale and consider each value as a discrete event over this time scale?

[georgebv]

I think I more or less understand what you are doing. You have a sample of some data and you want to model its tail. But, while you do so, you need to find the value to cut the sample into "body" and tail. I think before you do that you need to understand what model (distribution) you are choosing and why.

You shouldn't be using GPD or GEVD just because they model extreme values and visually look like a good fit for your data. Both of those models are backed with rigorous mathematical reasoning for why they are applicable in these specific cases:

• GEVD only for Block Maxima over time series
• GPD (or exponential in degenerate case) only for peaks-over-threshold (POT)

The goal of POT is to find smallest threshold which provides IID (independent identically distributed) data which follows GPD. If you cannot satisfy the IID criterion then GPD cannot be used as a model.

Since you don't have a signal (time series) I don't think EVA is the correct framework. In fact, I don't think pyextremes is the right tool here. It's primary goal is to help deal with time series which you don't even have. All you need is to fit a distribution and for that I recommend using scipy (which pyextremes uses under the hood).

You will need to answer the following questions:

• what does it mean for the occurrences of values above this threshold significantly increase
• what criteria would a good threshold need to meet
• what distribution is you data expected to follow - is it normal, multimodal, or something else
• if you can define where tail should be, what model should it follow

[Dillion]

Thanks for the explanation, I will read up more on the topic. I would like to understand abit more about the definition of signal as a time series and why it is so important in the EVA framework, can you direct me to some links/readings? Appreciate it

[georgebv]

I recommend "An Introduction to Statistical Modeling of Extreme Values" by Stuart Coles