-
Notifications
You must be signed in to change notification settings - Fork 13
/
CHANGELOG
340 lines (253 loc) · 12.4 KB
/
CHANGELOG
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
Datatest Changelog
==================
2021-01-03 (0.11.1)
-------------------
* Fixed validation, predicate, and difference handling of non-comparable
objects.
* Fixed bug in normalization of Queries from `squint` package.
* Changed failure output to imporove error reporting with Pandas accessors.
* Changed predicate failure message to quote code objects using backticks.
2020-12-17 (0.11.0)
-------------------
* Removed deprecated decorators: skip(), skipIf(), skipUnless() (use
unittest.skip(), etc. instead).
* Removed deprecated aliases Selector and ProxyGroup.
* Removed the long-deprecated "allowed" interface.
* Removed deprecated acceptances: "specific", "limit", etc.
* Removed deprecated `Select`, `Query`, and `Result` API. Use **squint**
instead:
- https://pypi.org/project/squint/
* Removed deprecated `get_reader()` function. Use **get-reader** instead:
- https://pypi.org/project/get-reader/
2020-12-17 (0.10.0)
-------------------
* Fixed bug where ValidationErrors were crashing pytest-xdist workers.
* Added tighter Pandas integration using Pandas' extension API.
After calling the new "register_accessors()" function, your existing
`DataFrame`, `Series`, `Index`, and `MultiIndex` objects will have
a validate() method that can be used instead of the validate()
function:
import padas as pd
import datatest as dt
dt.register_accessors() # <- Activate Pandas integration.
df = pd.DataFrame(...)
df[['A', 'B']].validate((str, int)) # <- New accessor method.
* Changed Pandas validation behavior:
* `DataFrame` and `Series`: These objects are treated as *sequences*
when they use a `RangeIndex` index (this is the default type
assigned when no index is specified). And they are treated as
*dictionaries* when they use an index of any other type--the
index values become the dictionary keys.
* `Index` and `MultiIndex`: These objects are treated as sequences.
* Changed `repr` behavior of Deviation to make timedeltas more readable.
* Added Predicate matching support for NumPy types `np.character`,
`np.integer`, `np.floating`, and `np.complexfloating`.
* Added improved NaN handling:
* Added NaN support to `accepted.keys()`, `accepted.args()`, and
`validate.interval()`.
* Improved existing NaN support for difference comparisons.
* Added how-to documentation for NaN handling.
* Added data handling support for `squint.Select` objects.
* Added deprecation warnings for soon-to-be-removed functions and classes:
* Added DeprecationWarning to `get_reader` function. This function
is now available from the **get-reader** package on PyPI:
https://pypi.org/project/get-reader/
* Added DeprecationWarning to `Select`, `Query`, and `Result` classes.
These classes will be deprecated in the next release but are now
available from the **squint** package on PyPI:
https://pypi.org/project/squint/
* Changed validate.subset() and validate.superset() behavior:
The semantics are now inverted. This behavior was flipped to more
closely match user expectations. The previous semantics were used
because they reflect the internal structure of datatest more precisely.
But these are implementation details that and they are not as important
as having a more intuitive API.
* Added temporary a warning when using the new subset superset methods
to alert users to the new behavior. This warning will be removed from
future versions of datatest.
* Added Python 3.9 and 3.10 testing and support.
* Removed Python 3.1 testing and support. If you were still using this
version of Python, please email me---this is a story I need to hear.
2019-06-03 (0.9.6)
------------------
* Changed acceptance API to make it both less verbose and
more expressive:
* Consolidated specific-instance and class-based acceptances
into a single interface.
* Added a new accepted.tolerance() method that subsumes the
behavior of accepted.deviation() by supporting Missing and
Extra quantities in addition to Deviation objects.
* Deprecated old methods:
======================= ==================================
Old Syntax New Syntax
======================= ==================================
accepted.specific(...) accepted(...)
accepted.missing() accepted(Missing)
accepted.extra() accepted(Extra)
*NO EQUIVALENT* accepted(CustomDifferenceClass)
accepted.deviation(...) accepted.tolerance(...)
accepted.limit(...) accepted.count(...)
*NO EQUIVALENT* accepted.count(..., scope='group')
======================= ==================================
Other methods--accepted.args(), accepted.keys(), etc.--remain
unchanged.
* Changed validation to generate Deviation objects for a broader
definition of quantitative values (like datetime objects)--not
just for subclasses of numbers.Number.
* Changed handling for pandas.Series objects to treat them as
sequences instead of mappings.
* Added handling for DBAPI2 cursor objects to automatically
unwrap single-value rows.
* Removed acceptance classes from datatest namespace--these were
inadvertently added in a previous version but were never part
of the documented API. They can still be referenced via the
`acceptances` module::
from datatest.acceptances import ...
2019-05-01 (0.9.5)
------------------
* Changed difference objects to make them hashable (can now be defined
as members of a set or used as dictionary keys).
* Added __slots__ to difference objects to reduce memory consumption.
* Changed name of Selector class to Select ("Selector" is now deprecated).
* Changed language and class names from "allowed & allowance" to
"accepted & acceptance" to bring datatest more inline with
manufacturing and engineering terminology. The existing "allowed"
API is now deprecated.
2019-04-21 (0.9.4)
------------------
* Added Python 3.8 testing and support.
* Added new validate methods (moved from how-to recipes into core module):
* Added approx() method to require for approximate numeric equality.
* Added fuzzy() method to require strings by approximate match.
* Added interval() method to require elements within a given interval.
* Added set(), subset(), and superset() methods for explicit membership
checking.
* Added unique() method to require unique elements.
* Added order() method to require elements by relative order.
* Changed default sequence validation to check elements by index position
rather than checking by relative order.
* Added fuzzy-matching allowance to allow strings by approximate match.
* Added Predicate class to formalize behavior--also provides inverse-matching
with the inversion operator (~).
* Added new methods to Query class:
* Added unwrap() to remove single-element containers and return their
unwrapped contents.
* Added starmap() to unpack grouped arguments when applying a function
to elements.
* Fixed improper use of assert statements with appropriate conditional
checks and error behavior.
* Added requirement class hierarchy (using BaseRequirement). This gives
users a cleaner way to implement custom validation behavior and makes
the underlying codebase easier to maintain.
* Changed name of ProxyGroup to RepeatingContainer.
* Changed "How To" examples to use the new validation methods.
2019-01-29 (0.9.3)
------------------
* Changed bundled pytest plugin to version 0.1.3:
* This update adds testing and support for latest versions of Pytest
and Python (now tested using Pytest 3.3 to 4.1 and Python 2.7 to 3.7).
* Changed handling for 'mandatory' marker to support older and newer
Pytest versions.
2018-08-08 (0.9.2)
------------------
* Improved data handling features and support for Python 3.7.
* Changed Query class:
* Added flatten() method to serialize dictionary results.
* Added to_csv() method to quickly save results as a CSV file.
* Changed reduce() method to accept "initializer_factory" as
an optional argument.
* Changed filter() method to support predicate matching.
* Added True and False as predicates to support "truth value testing" on
arbitrary objects (to match on truthy or falsy).
* Added ProxyGroup class for performing the same operations on groups of
objects at the same time (a common need when testing against reference
data).
* Changed Selector class keyword filtering to support predicate matching.
* Added handling to get_reader() to support datatest's Selector and Result
objects.
* Fixed get_reader() bug that prevented encoding-fallback recovery when
reading from StringIO buffers in Python 2.
2018-06-21 (0.9.1)
------------------
* Added impoved docstrings and other documentation.
* Changed bundled pytest plugin to version 0.1.2:
* Added handling for a 'mandatory' marker to support
incremental testing (stops session early when a mandatory
test fails).
* Added '--ignore-mandatory' option to continue tests
even when a mandatory test fails.
2018-04-29 (0.9.0)
------------------
* Added a bundled version of the pytest plugin to the base installation.
* Added universal composability for all allowances (using UNION and
INTERSECTION via "|" and "&" operators).
* Added ``allowed`` factory class to simplify allowance imports.
* Changed is_valid() to valid().
* Changed ValidationError to display differences in sorted order.
* Added Python 2 and 3 compatible get_reader() to quickly load
csv.reader-like interface for Unicode CSV, MS Excel, pandas.DataFrame,
DBF, etc.
* Added formal order of operations for allowance resolution.
* Added formal predicate object handling.
* Added Sphinx-tabs style docs for clear separation of pytest and
unittest style examples.
* Changed DataSource to Selector, DataQuery to Query, and DataResult to
Result.
2017-11-26 (0.8.3)
------------------
* Added module-level functions: validate() and is_valid().
* Changed DataQuery selections now default to a list type when no
outer-container is specified.
* Added DataQuery.apply() method for group-wise function application.
* Changed ValidationError repr to print a trailing comma with the last item
(for ease of copy-and-paste work flow).
* Changed sequence validation behavior provides more precise differences.
* Added truncation support for ValidationErrors with long lists of differences.
* Changed excess differences in allowed_specific() definitions no longer
trigger test failures.
* Added support for user-defined functions to narrow DataSource selections.
* Added traceback hiding for pytest.
* Fixed bug in DataQuery.map() method--now converts set types into lists.
2017-06-11 (0.8.2)
------------------
* Added Boolean composition for allowed_specific() context manager.
* Added proper __repr__() support to DataSource and DataQuery.
* Changed DataQuery so it fails early if bad "select" syntax is used or if
unknown columns are selected.
* Added __copy__() method to DataQuery.
* Changed parent class of differences so they no longer inherit from Exception
(this confused their intended use).
* Changed documentation structure for ease of reference.
2017-06-11 (0.8.1)
------------------
* Changed DataQuery select behavior to fail immediately when invalid syntax is
used (rather than later when attempting to execute the query).
* Changed error messages to better explain what went wrong.
2017-05-30 (0.8.0)
------------------
* Added a single, smarter assertValid() method.
* Removed old assertion methods.
* Added query optimization and a simpler and more expressive syntax.
* Changed allowances and errors to be more expressive.
* Added basic composability to some allowance classes using "&" and "|"
bit-wise operators.
2016-08-02 (0.7.0.dev2)
-----------------------
* Removed some of the internal magic and renames data assertions to more
clearly indicate their intended use.
* Changed data allowances to provide more consistent parameters and more
flexible usage.
* Added new method to assert unique values.
* Added full **fmtparams support for CSV handling.
* Fixed comparison and allowance behavior for None vs. zero.
2016-05-29 (0.6.0.dev1)
-----------------------
* First public release of rewritten code base.
Changelog Guidelines
====================
* Begin each section with the date followed by the version number in
parenthesis. Use the following format: "YYYY-MM-DD (x.y.z)".
* The initial bullet-point may provide a one-line description of the release.
* Following bullet-points should begin with "Added", "Changed", "Fixed", or
"Removed" when describing the notable changes.
* Limit lines to 80 character width.