-
Notifications
You must be signed in to change notification settings - Fork 7
/
Copy pathA-benchmarking.qmd
544 lines (371 loc) · 21.5 KB
/
A-benchmarking.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
---
editor:
markdown:
wrap: 72
---
# Benchmarking and temporal disagreggation {#a-bench .unnumbered}
## In this chapter
The sections below provide guidance on how to implement algorithms on
- Benchmarking of [seasonally adjusted data](#a-bench-sa)
- Benchmarking [high to low frequency](#a-bench-high-low) data
- [Temporal Disaggregation](#a-bench-tempd)
Using the [GUI](#t-gui-overview) with a [plug-in](#t-plug-ins) or
[rjd3bench package](#t-r-packs) package.
## Algorithms overview{#a-b-td-overview}
### Benchmarking {#t-plug-ins-bench}
| Method | GUI Plug-in for V 2.x | GUI Plug-in for V 3.x | In R rjd3bench |
|-----------------------|-----------------|-----------------|-----------------|
| Denton | ✔ | ✔ | ✔ |
| Cholette | ✔ | ✔ | ✔ |
| Cholette Multi-variate | ✔ | ✖ | ✔ |
| Cubic Splines | ✖ | ✔ | ✔ |
| GRP (Growth Rate Preservation) | ✖ | ✔ | ✔ |
| Calendarization | ✔ | ✖ | ✔ |
### Temporal Disaggregation
| Method | GUI Plug-in for V 2.x | GUI Plug-in for V 3.x | In R rjd3bench |
|---------------------------|---------------|---------------|---------------|
| Regression Models\* | ✔️ | ✔️ | ✔ |
| Model-based Denton | ✖ | ✔ | ✔ |
| ADL (Autoregressive Distributed Lag Models) | ✖ | ✖ | ✔ |
\*Regression models: several structures of residuals
- Ar1: Chow-Lin
- Rw: Fernandez
- RwAr1: Litterman
## Benchmarking seasonally adjusted data {#a-bench-sa}
The goal here is to enforce identical annual totals on the seasonally
adjusted series as on the raw or calendar adjusted series.
### Using the GUI
When running a seasonal adjustment process
1. With the pre-defined specifications the benchmarking functionality
is not applied by default following the *ESS Guidelines on Seasonal
Adjustment* (2024) recommendations. It means that once the user has
seasonally adjusted the series with a pre-defined specification the
*Benchmarking* node is empty. To execute benchmarking click on the
*Specifications* button and activate the checkbox in the
*Benchmarking* section.
2. Three parameters can be set here. *Target* specifies the target
variable for the benchmarking procedure. It can be either the
*Original* (the raw time series) or the *Calendar Adjusted* (the
time series adjusted for calendar effects). *Rho* is a value of the
AR(1) parameter (set between 0 and 1). By default it is set to 1.
Finally, *Lambda* is a parameter that relates to the weights in the
regression equation. It is typically equal to 0 (for an additive
decomposition), 0.5 (for a proportional decomposition) or 1 (for a
multiplicative decomposition). The default value is 1.
3. To launch the benchmarking procedure click on the **Apply** button.
The results are displayed in four panels. The top-left one compares
the original output from the seasonal adjustment procedure with the
result from applying a benchmarking to the seasonal adjustment. The
bottom-left panel highlights the differences between these two
results. The outcomes are also presented in a table in the top-right
panel. The relevant statistics concerning relative differences are
presented in the bottom-right panel.
4. Both pictures and the table can be copied the usual way (see the
[*Simple seasonal adjustment of a single time
series*](../case-studies/simplesa-single.html) scenario).
5. To export the result of the benchmarking procedure
(*benchmarking.result*) and the target data (*benchmarking.target*)
one needs to once execute the seasonal adjustment with benchmarking
6. Expand the "+" menu and choose an appropriate data format (here
Excel has been chosen). It is possible to save the results in TXT,
XLS, CSV, and CSV matrix formats. Note that the [available content
of the output depends on the output type](../theory/output.html).
7. Chose the output items that refer to the results from the
benchmarking procedure, move them to the window on the right and
click **OK**.
### In R with `rjd3x13` and `rjd3tramoseats`
When performing seasonal adjustment with `rjd3x13` and `rjd3tramoseats`,
the current (or default) specification has to be customized using the
function `rjd3toolkit::set_benchmarking` documented on this [GitHub
page](https://rjdverse.github.io/rjd3toolkit/reference/set_benchmarking.html)
```{r, echo=TRUE, eval=FALSE}
init_spec <- rjd3x13::spec_x13("RSA5c")
new_spec <- set_benchmarking(init_spec,
enabled = TRUE,
target = "Normal",
rho = 0.8,
lambda = 0.5,
forecast = FALSE,
bias = "None")
```
More information on R packages for JDemetra+ and installation procedures
is provided in [this chapter](#t-r-packs)
## Benchmarking with different frequencies {#a-bench-high-low}
These methods provide a high-frequency series (input series) modified so
that it fulfils a linear relationship, with another series of low
frequency (benchmark), both series measure the same target variable. An
example of the relation to be fulfilled could be that the low frequency
series (quarterly frequency) coincides with the quarterly sum of the
high frequency series (monthly frequency).
Multivariate benchmarking also forces contemporary linear relations
between high frequency series. If these relations do not exist,
benchmarking could be carried out for each series separately. Normally
contemporary relations are linear and the relations of aggregation are
also linear and the same for all series, so the contemporary relations
between low frequency series are fulfilled.
The benchmarking methods available in the benchmarking and time
disaggregation plug-in are: Denton, Cholette, and Cholette multivariate.
### Using the plug-in for GUI (version 2.x) {#a-bench-td-plugin}
Download the plug-in for GUI as explained [here](#t-plug-ins-bench) and
install it as detailed [here](#t-plug-ins-inst)
Once the plugin is installed, two more options appear in the Workspace
window: Benchmarking and Temporal Disaggregation.
#### Univariate: Denton and Cholette
To run Denton univariate case select:
Statistical **Methods** $\rightarrow$ Benchmarking $\rightarrow$ Denton
or Cholette
In both cases, a new window is displayed to launch one of the methods
with the series selected. In the upper left side, drag the high
frequency series from the Providers window and drop it in **Drop Series
here** and the low frequency series in **Drop Constraint here**.
#### Denton
In the top right of the screen, select the **Specifications** button to
set the specifications to apply each method. See below for a description
of the available options on Denton method:
1. **Type**: Aggregation function (Sum, Average, Last or First). This
forces the low-frequency series to match the aggregation function
selected of the high frequency series.
2. **Multiplicative**: if the checkbox is selected, the proportional
Denton method is applied. Otherwise, additive Denton is applied.
3. **Modified Denton**: if the checkbox is selected, the modified
Denton method is applied. Otherwise, original Denton is applied. It
is recommended to select it; as original Denton perform a special
treatment on the first observation.
4. **Differencing**: Number of regular differences. By default 1.
5. **Default frequency**: periodicity of the low frequency data. The
options are: Yearly, HalfYearly, QuadriMonthly, Quarterly, Bimonthly
and Monthly.
#### Cholette {#a-bench-high-low-cholette}
See below for a description of the available options on Cholette method:
1. **Type**: Aggregation function (Sum, Average, Last or First). This
forces the low-frequency series to match the aggregation function
selected of the high frequency series.
2. **Aggregation frequency**: periodicity of the low frequency data.
The options are: Yearly, HalfYearly, QuadriMonthly, Quarterly,
Bimonthly and Monthly.
3. **Rho**: value between $-1$ and $1$. It is the coefficient of an
AR($1$) model that follows the error term. The default value is $1$,
equivalent to applying Denton.
4. **Lambda**: value between $0$ and $1$. It is the parameter $\lambda$
of the following function to be minimized in Cholette method:
$$
\sum_t \left( \frac{x_t - z_t}{\left| z_t \right|^{\lambda}} - \rho \frac{x_{t-1} - z_{t-1}}{\left| z_{t-1} \right|^{\lambda}}\right)^2
$$
Usually lambda is $0$ or $1$ equivalent to applying additive
benchmarking and proportional benchmarking method respectively.
In both cases, Denton and Cholette methods, the output is a graph with
the original series and the benchmarked series. There is no table with
the results, but it is very easy to create one from the graph. Select
the graph and select copy, then paste the values in excel (control-V).
#### Multi-variate Cholette
The only multi-variate benchmarking method available for the version 2 plugin, is
multi-variate Cholette.
The input for this method are a set of time series with different
frequencies and a set of constraints, both contemporary and
intertemporal. The output is a new set of time series, that corresponds
to the former, now fulfilling the constraints.
To run multi-variate Cholette select *Statistical methods*
$\rightarrow$ *Benchmarking* $\rightarrow$ *Multi-variate Cholette*.
Then, a box appears, where we can drop time series.
The specification properties can be set by selecting *Window* $\rightarrow$
*Properties*. There are only three elements in the form: *Rho*, *Lambda* and
*Constraints*. The two first are analogous to their [univariate Cholette](#a-bench-high-low-cholette)
counterparts. The third one allows to set constraints, both contemporary and
intertemporal.
Clicking the three dots button from *Constraints*, opens the constraints list
box.
From this box, the set of constraints can be added, just typing the constraint
inside item and clicking *Add* button. The constraints must be written as
follows:
- $y = a_1 * x_1 + \dots + a_n * x_n$ where $y, x_1, \dots, x_n$ are same
frequency time series and $a_1, \dots, a_n$ are constant.
- $c = a_1 * x_1 + \dots + a_n * x_n$ where $x_1, \dots, x_n$ are same frequency
time series and $c, a_1, \dots, a_n$ are constant.
- $c = x_1 + \dots + x_n$ can be shortened as $c = x?$.
- $S = \mathrm{sum}(s)$ where $S$ is low-frequency and $s$ is high-frequency.
Note that any time series put on the left hand side can't appear on the right
hand side of any other constraint. This is because left hand side quantities
are fixed while right hand side quantities are adjusted so the equality holds.
The output are the benchmarked high-frequency time series, that can be found
in *Details* $\rightarrow$ *Benchmarked series*.
### Using the plug-in for GUI (version 3.x)
Practical use of the plug-in for v 3.x is quasi identical to the one in v2.x described
[above](#a-bench-td-plugin). Some methods are not available yet in v 3.x but the latter which contains Model
Based Denton not included in v2.x, as stated [here](#a-b-td-overview)
### In R with `rjd3bench`
Use the \[rjd3bench\](https://github.com/rjdverse/rjd3bench) package and
see its documentation pages. Browse its documentation on this [GitHub
page](https://rjdverse.github.io/rjd3bench/).
To get started browse the
[vignette](https://rjdverse.github.io/rjd3bench/articles/rjd3bench.html)
More information on R packages for JDemetra+ and installation procedures
is provided in [this chapter](#t-r-packs)
## Temporal Disaggregation {#a-bench-tempd}
These methods are used to disaggregate a series from low frequency to
high frequency. Temporal disaggregation methods developed in the plug-in
are Chow-Lin, Fernández and Litterman.
When there are high frequency related indicators, these methods provide
high frequency estimations for a series whose sums, averages, first or
last values are consistent with the observed low frequency series,
applying a regression model where it is assumed that the high frequency
series to be estimated follows a multiple regression with p related
series (indicators).
See Methods$\rightarrow$Temporal disaggregation for more theoretical
detail.
### Using the plug-in for GUI
Temporal disaggregation in the GUI is available with the same
[plug-in](#a-bench-td-plugin) as benchmarking (described in the sections
above)
To run Temporal Disaggregation methods select Temporal
disaggregation$\rightarrow$ Regression Model:
A new window is displayed to launch one of the methods with the series
selected. In the upper left side drag the low frequency series from the
Providers window and drop it in **Y box** and the proxy series or
indicator with high frequency series in **X box**.
In the top right of the screen, select **Specifications** to set the
specifications to apply each method. Here is a description of the
available options on Temporal Disaggregation methods:
1. **Estimation span**: Specifies the span (data interval) of the time
series to be used in the temporal disaggregation process. The user
can restrict the span. The common settings are:
| Option | Description (expected format) | |
|----------|-----------------------------------------------------|----------|
| All | default | |
| From | first observation included (yyyy-mm-dd) | |
| To | last observation included (yyyy-mm-dd) | |
| Between | interval \[from ; to\] included (yyyy-mm-dd to yyyy-mm-dd) | |
| First | number of observtions from the beginning of the series included (dynamic) (integer) | |
| Last | number of observations from the end of the series (dynamic)(integer) | |
| Excluding | excluding N first observation and P last observation from the computation,dynamic) (integer) | |
| Preliminary check | check to exclude highly problematic series e.g. the series with a number of identical observations and/or missing values above pre-specified threshold values. (True/False) | |
2. **Error**: determines the method to be applied and it refers to the
model that follows the error term.
| Option | Description |
|--------|----------------------------|
| Ar1 | Chow-Lin method (default) |
| Wn | Classical Regression model |
| Rw | Fernández |
| RwAr1 | Litterman |
| I2 | Integrated order 2 |
| I3 | Integrated order 3 |
3. **Parameter**: Coefficient of the AR(1) of the innovations model. It
has a value between -1 and 1. This parameter exists only if RWar1 or
Ar1 is selected in the error parameter.
4. **Constant**: a constant is included in the model if it is selected.
5. **Trend**: a linear trend is included in the model if it is
selected.
6. **Type**: Aggregation function (Sum, Average, Last or First). This
forces the low-frequency series to match the aggregation function
selected of the high frequency series.
7. **Default frequency**: it is the frequency of the output series.
8. **Advanced options**: These parameters are related to state space
model and the algorithm used to obtain the estimations.
8.1. **Diffuse regression coefficient**: Indicates if the
coefficients of the regression model are diffuse (T) or fixed
unknown (F, default).
Here are the results:
Select **Model**$\rightarrow$Summary to see the estimation of $rho$
(coefficient of the AR(1) model) and the coefficient of the regression
model. Additionally the BIC, AIC and AICC. It is also showed the
variance decomposition in Indicators and Smoothing. Ideally, if the
indicator adequately approximates the aggregate in the observable domain
(low frequency model), the residuals of the low frequency model will be
small and the indicator term will dominate. \\
To confirm that the model works well, select
**Model**$\rightarrow$Residuals$\rightarrow$Statistics and see the tests
on the residuals of the model:
Select MainResults$\rightarrow$Table to obtain the disaggregated series
and standard deviation.
Select **MainResults**$\rightarrow$Chart to see a graph of the
disaggregated series and the confidence interval.
### In R with `rjd3bench`
Use the \[rjd3bench\](https://github.com/rjdverse/rjd3bench) package and
see its documentation pages. Browse its documentation on this [GitHub
page](https://rjdverse.github.io/rjd3bench/).
To get started browse the
[vignette](https://rjdverse.github.io/rjd3bench/articles/rjd3bench.html)
More information on R packages for JDemetra+ and installation procedures
is provided in [this chapter](#t-r-packs)
#### Temporal Disaggregation
To perform Temporal Disaggregation methods use the function
**temporaldisaggregation**:
```{r, echo = TRUE, eval = FALSE}
output <- rjd3bench::temporaldisaggregation(
series = y, indicators = x, model = "Rw", freq = 12,
conversion = "Average", diffuse.algorithm = "Diffuse"
)
```
The input parameters are the same as in the GUI, see the R Documentation
of the rjd3bench package for the description.
The output is a list containing 3 elements:
1. **Regression**: contains information about:
- Type of method applied:
```{r,eval=FALSE, include=TRUE}
output$regression$type
```
- The model (coefficient estimation, standard deviation and
T-statistic):
```{r,eval=FALSE, include=TRUE}
output$regression$model
```
- Conversion:Aggregation function (Sum, Average, Last or First):
```{r,eval=FALSE, include=TRUE}
output$regression$conversion
```
2. **Estimation**: contains information about:
- The disaggregated series:
```{r,eval=FALSE, include=TRUE}
output$estimation$disagg
```
- The standard deviation of the disaggregated series:
```{r,eval=FALSE, include=TRUE}
output$estimation$edisagg
```
- The regressor effect:
```{r,eval=FALSE, include=TRUE}
output$estimation$regeffect
```
- The smoothing part:
```{r,eval=FALSE, include=TRUE}
output$estimation$smoothingpart
```
- The $rho$ estimation (coefficient of the AR(1) model): This
parameter exists only if RWar1 or Ar1 is selected in the model.
```{r,eval=FALSE, include=TRUE}
output$estimation$parameter
```
- The standard deviation of the AR(1) coefficient:
```{r,eval=FALSE, include=TRUE}
output$estimation$eparameter
```
3. **Likelihood**: Contains information about the loglikelihood(ll),
sum of squares of the residuals of the model (ssq), number of
observations (nobs), number of parameters to be estimated (nparams),
degrees of freedom (df), Akaike Information Criteria (aic), Akaike
Information Criteria Corrected (aicc), Bayesian Information Criteria
(bic), Bayesian Information Criteria Corrected (bic2).