-
-
Notifications
You must be signed in to change notification settings - Fork 159
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #2619 from jdebacker/pr-2453
Add TAXSIM-35 Validation: To replace PR #2453
- Loading branch information
Showing
22 changed files
with
1,275 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,133 @@ | ||
Validation of Tax-Calculator against Internet TAXSIM-35 | ||
======================================================= | ||
|
||
The general cross-model validation process described | ||
[here](https://github.com/PSLmodels/Tax-Calculator/blob/master/taxcalc/validation/README.md#validation-of-tax-calculator-logic) | ||
is being executed in this directory using | ||
[TAXSIM-35](https://users.nber.org/~taxsim/taxsim27/). | ||
|
||
We are in the process of comparing Tax-Calculator and TAXSIM-35 | ||
results generated from several assumption sets in the `taxsim_input.py` | ||
script for years beginning with 2018. Each INPUT file is | ||
used to generate a TAXSIM-35 OUTPUT file by uploading it to the | ||
TAXSIM-35 website and requesting detailed intermediate calculations. | ||
And each INPUT file is translated into a CSV-formatted input file that | ||
is read by the Tax-Calculator `tc` CLI tool to generate output that is | ||
then transformed into an OUTPUT file having the TAXSIM-35 format. | ||
Finally, these two OUTPUT files are compared using the `main_comparison.py` | ||
script. See the `tests_35.py` script in this directory for more details. | ||
|
||
The following results are for INPUT files containing 100,000 | ||
randomly-generated filing units for a given year. The random sampling | ||
is such that a different sample is drawn for each year. In each INPUT | ||
file three state-related variables are set to zero for every filing | ||
unit, one variable specifies the year, and another specifies a filing | ||
unit id, which leaves twenty-two input variables that are set to | ||
randomly-generated values. | ||
|
||
In order to handle known differences in assumptions between the two | ||
models, we use the `taxsim_emulation.json` "reform" file to make | ||
Tax-Calculator operate like TAXSIM-35. See the | ||
[`taxsim_emulation.json` | ||
file](https://github.com/PSLmodels/Tax-Calculator/blob/master/taxcalc/validation/taxsim35/taxsim_emulation.json) | ||
for details. | ||
|
||
In the following results, when we say "same results" we mean that the | ||
federal individual income tax liabilities and payroll tax liabilities | ||
being compared have differences of no larger than one cent. | ||
|
||
For information on the variable names illustrated in `taxsim_input.py`, | ||
the document that generates data for input into TAXSIM-35, see the TAXSIM-35 website listed above. | ||
|
||
|
||
Instructions | ||
------------------ | ||
1. Navigate to `taxcalc/validation/taxsim35` and run the Python script `tests_35.py`. | ||
2. If you would like to generate new input files and and get new files from TAXSIM-35, | ||
just delete all of the `.in.out-taxsim` files. On Mac/Linux, this can be done with | ||
`rm -f *.in.out-taxsim`. | ||
|
||
|
||
Troubleshooting | ||
------------------ | ||
If the TAXSIM-35 validation code throws errors such as `.in files not found`, | ||
`.out files not found` or that any parameter within `policy_current_law.json` | ||
does not exist, please try these 2 steps: | ||
|
||
1. Make sure that the `taxcalc` conda package is installed | ||
2. If you have Tax-Calculator downloaded locally, navigate to the root directory | ||
and run `pip install -e .` This will install the current source code into the `taxcalc` | ||
CLI. | ||
|
||
|
||
Validation Results | ||
------------------ | ||
|
||
**a18 ASSUMPTION SET**: | ||
|
||
2018 INPUT file that specifies the first twelve of the TAXSIM-35 | ||
input variables, which include demographic variables and labor income, | ||
but sets to zero all the TAXSIM-35 input variables numbered from 13 | ||
through 27. | ||
|
||
Validation results using the then current-version of TAXSIM-35 on these dates: | ||
|
||
**b18 ASSUMPTION SET**: | ||
|
||
2018 INPUT file that specifies the first twenty-one of the TAXSIM-35 | ||
input variables, which include demographic variables, labor income, | ||
capital income, and federally-taxable benefits, but set to zero all | ||
the other six TAXSIM-35 input variables except variables 28-35, | ||
which are the variables representing the new QBI-related variables. | ||
Two of those six are always set to zero because they specify transfer income | ||
that is not taxed under the federal income tax or because they specify rent paid that | ||
does not affect federal income tax liability. Three of the remaining | ||
four input variables are itemized expense amounts and the fourth is | ||
child-care expenses. | ||
|
||
Validation results using the then current-version of TAXSIM-35 on these dates: | ||
|
||
**c18 ASSUMPTION SET**: | ||
|
||
2018 INPUT file that specifies all the non-state TAXSIM-35 input | ||
variables to be randomly generated values. | ||
|
||
Validation results using the then current-version of TAXSIM-35 on these dates: | ||
|
||
**a19 ASSUMPTION SET**: | ||
|
||
2019 INPUT file that specifies the first twelve of the TAXSIM-35 | ||
input variables, which include demographic variables and labor income, | ||
but sets to zero all the TAXSIM-35 input variables numbered from 13 | ||
through 27. (This is the same logic as used to generate the **a17** | ||
sample except that a different stream of random numbers is used so that | ||
the 100,000 filing units are completely different.) | ||
|
||
Validation results using the then current-version of TAXSIM-35 on these dates: | ||
|
||
**b19 ASSUMPTION SET**: | ||
|
||
2019 INPUT file that specifies the first twenty-one of the TAXSIM-35 | ||
input variables, which include demographic variables, labor income, | ||
capital income, and federally-taxable benefits, but set to zero all | ||
the other six TAXSIM-35 input variables except variables 28-35, | ||
which are the variables representing the new QBI-related variables. | ||
Two of those six are always set to zero because they specify transfer income | ||
that is not taxed under the federal income tax or because they specify rent paid that | ||
does not affect federal income tax liability. Three of the remaining | ||
four input variables are itemized expense amounts and the fourth is | ||
child-care expenses. (This is the same logic as used to generate the | ||
**b17** sample except that a different stream of random numbers is | ||
used so that the 1,000 filing units are completely different.) | ||
|
||
Validation results using the then current-version of TAXSIM-35 on these dates: | ||
|
||
**c19 ASSUMPTION SET**: | ||
|
||
2019 INPUT file that specifies all the non-state TAXSIM-35 input | ||
variables to be randomly generated values. (This is the same logic as | ||
used to generate the **c17** sample except that a different stream of | ||
random numbers is used so that the 100,000 filing units are completely | ||
different.) | ||
|
||
Validation results using the then current-version of TAXSIM-35 on these dates: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
,# of differing records,max_diff,max_diff_index,max_diff_taxsim_val,max_diff_taxcalc_val | ||
fiitax,49,-0.00999999999999801,12135,-20.98,-20.99 | ||
siitax,0,0.0,no diff,no diff,no diff | ||
fica,0,0.0,no diff,no diff,no diff | ||
frate,0,0.0,no diff,no diff,no diff | ||
srate,0,0.0,no diff,no diff,no diff | ||
ficar,124,0.8999999999999999,172,2.9,3.8 | ||
v10,0,0.0,no diff,no diff,no diff | ||
v11,0,0.0,no diff,no diff,no diff | ||
v12,0,0.0,no diff,no diff,no diff | ||
v13,100000,-26600.0,12,26600.0,0.0 | ||
v14,0,0.0,no diff,no diff,no diff | ||
v15,0,0.0,no diff,no diff,no diff | ||
v16,0,0.0,no diff,no diff,no diff | ||
v17,0,0.0,no diff,no diff,no diff | ||
v18,0,0.0,no diff,no diff,no diff | ||
v19,0,0.0,no diff,no diff,no diff | ||
v20,0,0.0,no diff,no diff,no diff | ||
v21,0,0.0,no diff,no diff,no diff | ||
v22,0,0.0,no diff,no diff,no diff | ||
v23,0,0.0,no diff,no diff,no diff | ||
v24,0,0.0,no diff,no diff,no diff | ||
v25,46,0.00999999999999801,12135,20.98,20.99 | ||
v26,0,0.0,no diff,no diff,no diff | ||
v27,0,0.0,no diff,no diff,no diff | ||
v28,0,0.0,no diff,no diff,no diff |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
,# of differing records,max_diff,max_diff_index,max_diff_taxsim_val,max_diff_taxcalc_val | ||
fiitax,71,-0.010000000000005116,12332,-119.72,-119.73 | ||
siitax,0,0.0,no diff,no diff,no diff | ||
fica,0,0.0,no diff,no diff,no diff | ||
frate,0,0.0,no diff,no diff,no diff | ||
srate,0,0.0,no diff,no diff,no diff | ||
ficar,119,0.8999999999999999,2226,2.9,3.8 | ||
v10,0,0.0,no diff,no diff,no diff | ||
v11,0,0.0,no diff,no diff,no diff | ||
v12,0,0.0,no diff,no diff,no diff | ||
v13,100000,-27000.0,9,27000.0,0.0 | ||
v14,0,0.0,no diff,no diff,no diff | ||
v15,0,0.0,no diff,no diff,no diff | ||
v16,0,0.0,no diff,no diff,no diff | ||
v17,0,0.0,no diff,no diff,no diff | ||
v18,0,0.0,no diff,no diff,no diff | ||
v19,0,0.0,no diff,no diff,no diff | ||
v20,0,0.0,no diff,no diff,no diff | ||
v21,0,0.0,no diff,no diff,no diff | ||
v22,0,0.0,no diff,no diff,no diff | ||
v23,0,0.0,no diff,no diff,no diff | ||
v24,0,0.0,no diff,no diff,no diff | ||
v25,71,0.010000000000005116,4164,119.72,119.73 | ||
v26,0,0.0,no diff,no diff,no diff | ||
v27,0,0.0,no diff,no diff,no diff | ||
v28,0,0.0,no diff,no diff,no diff |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
,# of differing records,max_diff,max_diff_index,max_diff_taxsim_val,max_diff_taxcalc_val | ||
fiitax,100000,-130262.98999999999,75368,306837.54,176574.55 | ||
siitax,0,0.0,no diff,no diff,no diff | ||
fica,67638,-0.3000000000029104,14298,38376.98,38376.68 | ||
frate,46049,-55.0,91965,40.0,-15.0 | ||
srate,0,0.0,no diff,no diff,no diff | ||
ficar,0,0.0,no diff,no diff,no diff | ||
v10,99931,-350001.9800000002,99037,1402920.62,1052918.64 | ||
v11,0,0.0,no diff,no diff,no diff | ||
v12,5,-10980.150000000001,86601,39100.0,28119.85 | ||
v13,100000,-26600.0,12,26600.0,0.0 | ||
v14,0,0.0,no diff,no diff,no diff | ||
v15,0,0.0,no diff,no diff,no diff | ||
v16,0,0.0,no diff,no diff,no diff | ||
v17,0,0.0,no diff,no diff,no diff | ||
v18,100000,-402620.99,27455,653482.24,250861.25 | ||
v19,100000,-134001.72999999998,27455,181167.43,47165.7 | ||
v20,0,0.0,no diff,no diff,no diff | ||
v21,0,0.0,no diff,no diff,no diff | ||
v22,9067,8500.0,888,0.0,8500.0 | ||
v23,8,2157.88,42055,0.0,2157.88 | ||
v24,0,0.0,no diff,no diff,no diff | ||
v25,0,0.0,no diff,no diff,no diff | ||
v26,100000,-350000.0,44030,685450.0,335450.0 | ||
v27,5513,13628.83,99719,0.0,13628.83 | ||
v28,100000,-130941.73,27455,178107.43,47165.7 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
,# of differing records,max_diff,max_diff_index,max_diff_taxsim_val,max_diff_taxcalc_val | ||
fiitax,100000,-138933.06,58416,284963.75,146030.69 | ||
siitax,0,0.0,no diff,no diff,no diff | ||
fica,67719,-0.3000000000029104,21816,39016.15,39015.85 | ||
frate,48029,-64.44,82573,49.44,-15.0 | ||
srate,0,0.0,no diff,no diff,no diff | ||
ficar,0,0.0,no diff,no diff,no diff | ||
v10,99999,-380001.98,99411,1619677.51,1239675.53 | ||
v11,0,0.0,no diff,no diff,no diff | ||
v12,13,-24155.15,52779,46750.0,22594.85 | ||
v13,100000,-27000.0,9,27000.0,0.0 | ||
v14,0,0.0,no diff,no diff,no diff | ||
v15,0,0.0,no diff,no diff,no diff | ||
v16,0,0.0,no diff,no diff,no diff | ||
v17,0,0.0,no diff,no diff,no diff | ||
v18,100000,-411751.91000000003,34865,645007.02,233255.11 | ||
v19,100000,-140755.06,58416,286785.75,146030.69 | ||
v20,0,0.0,no diff,no diff,no diff | ||
v21,0,0.0,no diff,no diff,no diff | ||
v22,10340,8500.0,2256,0.0,8500.0 | ||
v23,28,3397.83,82573,0.0,3397.83 | ||
v24,0,0.0,no diff,no diff,no diff | ||
v25,0,0.0,no diff,no diff,no diff | ||
v26,100000,-378605.36,56063,753678.58,375073.22 | ||
v27,6590,16327.24,37042,0.0,16327.24 | ||
v28,100000,-137185.06,58416,283215.75,146030.69 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
,# of differing records,max_diff,max_diff_index,max_diff_taxsim_val,max_diff_taxcalc_val | ||
fiitax,100000,-132150.84999999998,90016,150067.43,17916.58 | ||
siitax,0,0.0,no diff,no diff,no diff | ||
fica,67746,-0.2900000000008731,573,18578.54,18578.25 | ||
frate,46631,56.85,12070,36.74,93.59 | ||
srate,0,0.0,no diff,no diff,no diff | ||
ficar,0,0.0,no diff,no diff,no diff | ||
v10,99919,-350001.9800000002,69402,1546224.37,1196222.39 | ||
v11,0,0.0,no diff,no diff,no diff | ||
v12,1,-11650.0,9032,27200.0,15550.0 | ||
v13,32064,-26600.0,12,26600.0,0.0 | ||
v14,0,0.0,no diff,no diff,no diff | ||
v15,0,0.0,no diff,no diff,no diff | ||
v16,0,0.0,no diff,no diff,no diff | ||
v17,1414,26000.0,661,0.0,26000.0 | ||
v18,100000,-393130.3,81574,671568.48,278438.18 | ||
v19,100000,-133445.18,81574,187859.34,54414.16 | ||
v20,0,0.0,no diff,no diff,no diff | ||
v21,0,0.0,no diff,no diff,no diff | ||
v22,9041,8500.0,4303,0.0,8500.0 | ||
v23,34,3799.6,73248,0.0,3799.6 | ||
v24,16,-600.19,9032,600.19,0.0 | ||
v25,0,0.0,no diff,no diff,no diff | ||
v26,100000,-355450.88999999996,17060,570001.09,214550.2 | ||
v27,3942,10623.65,14603,0.0,10623.65 | ||
v28,100000,-131575.16999999998,81574,185989.34,54414.17 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
,# of differing records,max_diff,max_diff_index,max_diff_taxsim_val,max_diff_taxcalc_val | ||
fiitax,100000,-141609.81999999998,71749,164446.08,22836.26 | ||
siitax,0,0.0,no diff,no diff,no diff | ||
fica,67822,-0.3000000000029104,48837,38263.66,38263.36 | ||
frate,48751,-75.15,69110,60.15,-15.0 | ||
srate,0,0.0,no diff,no diff,no diff | ||
ficar,0,0.0,no diff,no diff,no diff | ||
v10,99997,-380001.98,43179,1390514.98,1010513.0 | ||
v11,0,0.0,no diff,no diff,no diff | ||
v12,11,-15775.25,19813,31450.0,15674.75 | ||
v13,32257,-27000.0,9,27000.0,0.0 | ||
v14,0,0.0,no diff,no diff,no diff | ||
v15,0,0.0,no diff,no diff,no diff | ||
v16,0,0.0,no diff,no diff,no diff | ||
v17,1834,27000.0,595,0.0,27000.0 | ||
v18,100000,-411471.92000000004,24420,683044.92,271573.0 | ||
v19,100000,-142118.92,57949,308485.83,166366.91 | ||
v20,0,0.0,no diff,no diff,no diff | ||
v21,0,0.0,no diff,no diff,no diff | ||
v22,10166,8500.0,3265,0.0,8500.0 | ||
v23,78,4640.22,36893,0.0,4640.22 | ||
v24,12,-820.96,30022,1200.0,379.04 | ||
v25,0,0.0,no diff,no diff,no diff | ||
v26,100000,-376000.93,71828,506700.93,130700.0 | ||
v27,4416,10468.79,27286,0.0,10468.79 | ||
v28,100000,-138202.08,75423,290379.91,152177.83 |
Oops, something went wrong.