CLN: replace %s syntax with .format in pandas.io #24454

makeajourney · 2018-12-27T21:36:50Z

progress towards #16130

passes git diff upstream/master -u -- "*.py" | flake8 --diff

jreback

can you use named parameters rather than positional, esp where there are more string to replace

jreback · 2018-12-27T21:42:54Z

pandas/io/parsers.py

@@ -2236,10 +2239,11 @@ def __init__(self, f, **kwds):
            raise ValueError('Only length-1 decimal markers supported')

        if self.thousands is None:
-            self.nonnum = re.compile('[^-^0-9^%s]+' % self.decimal)
+            self.nonnum = re.compile(
+                '[^-^0-9^{decimal}]+'.format(decimal=self.decimal))


can you make both of the raw string (r'....')

can you add this

jreback · 2018-12-27T21:45:16Z

pandas/io/pytables.py

-                "value->%s,format->%s,append->%s,kwargs->%s]"
-                % (t, group, type(value), format, append, kwargs)
-            )
+                "cannot properly create the storer for: [{}] [group->{},"


can you use named parameters here

jreback · 2018-12-27T21:46:51Z

pandas/io/pytables.py

@@ -1590,7 +1594,7 @@ def __unicode__(self):
                     self.axis,
                     self.pos,
                     self.kind)))
-        return "name->%s,cname->%s,axis->%s,pos->%s,kind->%s" % temp
+        return "name->{},cname->{},axis->{},pos->{},kind->{}".format(*temp)


here as well, maybe better to write as
','.join(("[{}->{}".format(key, value) for key, value in zip(['name', 'cname', 'axis', 'pos', 'kind'], temp)

codecov · 2018-12-27T23:54:33Z

Codecov Report

Merging #24454 into master will decrease coverage by 49.3%.
The diff coverage is 64.19%.

@@             Coverage Diff             @@
##           master   #24454       +/-   ##
===========================================
- Coverage   92.31%      43%   -49.31%     
===========================================
  Files         163      163               
  Lines       51987    51987               
===========================================
- Hits        47990    22356    -25634     
- Misses       3997    29631    +25634

Flag	Coverage Δ
#multiple	`?`
#single	`43% <64.19%> (ø)`	⬆️

Impacted Files	Coverage Δ
pandas/io/packers.py	`14.82% <0%> (-73.26%)`	⬇️
pandas/io/sas/sas7bdat.py	`0% <0%> (-91.17%)`	⬇️
pandas/io/parsers.py	`48.51% <5.88%> (-46.87%)`	⬇️
pandas/io/pytables.py	`91.38% <85%> (-0.93%)`	⬇️
pandas/io/formats/latex.py	`0% <0%> (-100%)`	⬇️
pandas/core/categorical.py	`0% <0%> (-100%)`	⬇️
pandas/io/sas/sas_constants.py	`0% <0%> (-100%)`	⬇️
pandas/tseries/plotting.py	`0% <0%> (-100%)`	⬇️
pandas/tseries/converter.py	`0% <0%> (-100%)`	⬇️
pandas/io/formats/html.py	`0% <0%> (-98.65%)`	⬇️
... and 125 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3086e0a...153ae0d. Read the comment docs.

codecov · 2018-12-27T23:54:33Z

Codecov Report

Merging #24454 into master will increase coverage by <.01%.
The diff coverage is 84.52%.

@@            Coverage Diff             @@
##           master   #24454      +/-   ##
==========================================
+ Coverage   92.38%   92.38%   +<.01%     
==========================================
  Files         166      166              
  Lines       52321    52321              
==========================================
+ Hits        48337    48338       +1     
+ Misses       3984     3983       -1

Flag	Coverage Δ
#multiple	`90.8% <39.28%> (ø)`	⬆️
#single	`43.06% <65.47%> (-0.01%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/packers.py	`88.21% <0%> (ø)`	⬆️
pandas/io/sas/sas7bdat.py	`91.16% <33.33%> (ø)`	⬆️
pandas/io/pytables.py	`92.31% <85.71%> (ø)`	⬆️
pandas/io/parsers.py	`95.4% <94.11%> (ø)`	⬆️
pandas/util/testing.py	`88.09% <0%> (+0.09%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d46d2a5...f249d93. Read the comment docs.

jreback · 2018-12-27T23:58:22Z

pandas/io/pytables.py

                        "Consider using min_itemsize to preset the sizes on "
-                        "these columns" % (itemsize, self.cname, c.itemsize))
+                        "these columns".format(


many locations still that need to use kwargs

jreback · 2018-12-27T23:58:51Z

pandas/io/pytables.py

@@ -1840,7 +1849,7 @@ def create_for_block(
        """ return a new datacol with the block i """

        if cname is None:
-            cname = name or 'values_block_%d' % i
+            cname = name or 'values_block_{idx}'.format(idx=i)


for a single replacement ok with NOT using kwargs

jreback · 2018-12-27T23:59:07Z

pandas/io/pytables.py

@@ -1876,7 +1885,8 @@ def __unicode__(self):
                     self.dtype,
                     self.kind,
                     self.shape)))
-        return "name->%s,cname->%s,dtype->%s,kind->%s,shape->%s" % temp
+        return ("name->{},cname->{},dtype->{},kind->{},"


use the same template as above

jreback · 2018-12-28T00:00:07Z

let's be consistent about this. a single formatted string need not be named, multiple need to be named.

WillAyd · 2019-01-02T16:35:43Z

pandas/io/parsers.py

                    else:
-                        col = '%s.%d' % (col, cur_count)
+                        col = '{col}.{cnt}'.format(col=col, cnt=cur_count)


Would be nice to use variable names consistently across modules, so count instead of int and column instead of col, etc...

WillAyd · 2019-01-02T16:36:13Z

pandas/io/parsers.py

@@ -1571,8 +1572,8 @@ def _get_name(icol):
                return icol

            if col_names is None:
-                raise ValueError(('Must supply column order to use %s as '
-                                  'index') % str(icol))
+                raise ValueError(('Must supply column order to use {icol!s} '


Do we need to be explicit about type in this and subsequent example?

if you said about '!s', yes. !s is replacing the str function not '%s'.

WillAyd · 2019-01-02T16:37:50Z

pandas/io/parsers.py

-                raise ValueError("Unable to convert column %s to "
-                                 "type %s" % (column, cast_type))
+                raise ValueError(
+                    "Unable to convert column {column} to type {type}".format(


Does using type here present any potential conflict with the built-in? Maybe renamed to type_ to err on the side of caution?

jbrockmendel · 2019-01-06T16:15:39Z

FWIW in case we someday want to lint for this, searching for "(?<!\d)\s%(?=\s)(?! \d)" shows 768 matches across the codebase. That pattern doesn't get rid of all the mod ops, but cuts it down quite a bit.

WillAyd · 2019-01-09T20:13:55Z

@makeajourney can you merge master and update comments?

pep8speaks · 2019-01-10T00:03:08Z

Hello @makeajourney! Thanks for updating the PR.

Cheers ! There are no PEP8 issues in this Pull Request. 🍻

Comment last updated on January 11, 2019 at 02:13 Hours UTC

makeajourney · 2019-01-11T01:31:50Z

CI failed only with python2.7. I have no idea about this problem.

makeajourney · 2019-01-11T02:12:00Z

@jreback Can I divide this pull-request into smaller units? It's too hard to find the problem because too many files changed.

progress towards pandas-dev#16130

This reverts commit 494ee10.

jreback · 2019-01-11T02:51:35Z

yes that would be best

jreback requested changes Dec 27, 2018

View reviewed changes

jreback added the Code Style Code style, linting, code_checks label Dec 27, 2018

jreback requested changes Dec 27, 2018

View reviewed changes

WillAyd reviewed Jan 2, 2019

View reviewed changes

makeajourney force-pushed the issue-16130-io branch from 2f3648b to c509318 Compare January 10, 2019 00:03

makeajourney added 9 commits January 11, 2019 11:12

CLN: replace %s syntax with .format in pandas.io

0dbffcc

progress towards pandas-dev#16130

Using named parameters

1410a88

moved position of closing parenthesis

d6112eb

Changed unnamed parameter to named

df295bf

changed parameter name

fe14b4c

add raw string symbol to string which is for regular expression

bc58bef

column variable is displayed as integer

548e121

Revert "column variable is displayed as integer"

3766524

This reverts commit 494ee10.

Add unicode prefix

f249d93

makeajourney force-pushed the issue-16130-io branch from 1f305c8 to f249d93 Compare January 11, 2019 02:13

makeajourney closed this Jan 11, 2019

makeajourney deleted the issue-16130-io branch January 11, 2019 02:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CLN: replace %s syntax with .format in pandas.io #24454

CLN: replace %s syntax with .format in pandas.io #24454

makeajourney commented Dec 27, 2018

jreback left a comment

jreback Dec 27, 2018

jreback Jan 10, 2019

jreback Dec 27, 2018

jreback Dec 27, 2018

codecov bot commented Dec 27, 2018

codecov bot commented Dec 27, 2018 •

edited

Loading

jreback Dec 27, 2018

jreback Dec 27, 2018

jreback Dec 27, 2018

jreback commented Dec 28, 2018

WillAyd Jan 2, 2019

WillAyd Jan 2, 2019

makeajourney Jan 9, 2019

WillAyd Jan 2, 2019

jbrockmendel commented Jan 6, 2019

WillAyd commented Jan 9, 2019

pep8speaks commented Jan 10, 2019 •

edited

Loading

makeajourney commented Jan 11, 2019

makeajourney commented Jan 11, 2019

jreback commented Jan 11, 2019

CLN: replace %s syntax with .format in pandas.io #24454

CLN: replace %s syntax with .format in pandas.io #24454

Conversation

makeajourney commented Dec 27, 2018

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Dec 27, 2018

Codecov Report

codecov bot commented Dec 27, 2018 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Dec 28, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbrockmendel commented Jan 6, 2019

WillAyd commented Jan 9, 2019

pep8speaks commented Jan 10, 2019 • edited Loading

Comment last updated on January 11, 2019 at 02:13 Hours UTC

makeajourney commented Jan 11, 2019

makeajourney commented Jan 11, 2019

jreback commented Jan 11, 2019

codecov bot commented Dec 27, 2018 •

edited

Loading

pep8speaks commented Jan 10, 2019 •

edited

Loading