Replace old string formatting syntax with f-strings #29547

ShaharNaveh · 2019-11-11T18:20:58Z

Since we no longer support python 3.5, we can now use the new f-strings instead of the old .format() ( and obviously the % formatting).

Notes:

Don't forget to link this issue in your pull request's body message , simply paste this https://github.com/pandas-dev/pandas/issues/29547 in your pull request's body message.
If any of your changed files are related to Replace "foo!r" to "repr(foo)" syntax #29886 , please make sure to link your pull request to that issue as well, simply paste this https://github.com/pandas-dev/pandas/issues/29886 in your pull request's body message as well.
Please comment what you are planning to work on, so we won't do double work.
If a file/files that should be marked as done, is not marked, please comment letting me know.

To check what files still needs to be fixed in the pandas directory:

grep -l -R '%s'  --include=*.{py,pyx} pandas/
grep -l -R '%d' --include=*.{py,pyx} pandas/
grep -l -R '\.format(' --include=*.{py,pyx} pandas/

All of the above can also be used as a one liner:

grep -l -R -e '%s' -e '%d' -e '\.format(' --include=*.{py,pyx} pandas/

Tip:

If you want to see the line number of the occurrence, replace the -l with -n
for example:

grep -n -R '%s' --include=*.{py,pyx} pandas/

The current list is:

NOTE:

The list may change as files are moved/renamed constantly.

Inhereted files and commands from this PR.

The text was updated successfully, but these errors were encountered:

ShaharNaveh · 2019-11-11T23:07:09Z

yashukla · 2019-11-12T06:45:57Z

I'll take:

pandas/tests/indexes/test_base.py
pandas/tests/indexes/test_category.py
pandas/tests/indexes/test_common.py
pandas/tests/indexes/test_numeric.py

to start, if that's alright!

SaturnFromTitan · 2019-11-12T09:57:47Z

Hi @MomIsBestFriend Can you recommend any tools for this conversion? A quick look gave me these:

I have no experience with either of them, but they could be very helpful here

ShaharNaveh · 2019-11-12T10:31:11Z

Hello @SaturnFromTitan , I personally sometimes use pyupgrade but only when the file contain only a few outdated string formats in it. Then I look at the changes and fix if pyupgrade got something wrong.

When they're files with alot of occurrences I go for the "complex" ones manually (e.g '%.2f' % my_float) and let it deal with the common ones, usually it gets it right.

Also, some of the changes will make the changed file non pep8 compatible, so there's a need to fix that as well, otherwise it will not pass the tests.

ShaharNaveh · 2019-11-14T21:19:13Z

Will take next:

pandas/compat/__init__.py
pandas/compat/numpy/function.py
pandas/compat/numpy/__init__.py
pandas/compat/_optional.py

yashukla · 2019-11-15T01:54:15Z

I'll take:

pandas/core/reshape/concat.py
pandas/core/reshape/melt.py
pandas/core/reshape/merge.py
pandas/core/reshape/pivot.py
pandas/core/reshape/reshape.py
pandas/core/reshape/tile.py

What are everyone's thoughts on tagging this as a good first issue? It should apply to most of the files here. The changes that need to be made are usually only a few lines or so per file, and whoever is making the changes doesn't need to worry too much about affecting other parts of the code (since the end function performed is the same).

I'm picturing a setup similar to #28926.

…) (pandas-dev#29579)

lucassa3 · 2019-11-19T01:22:55Z

f-string replacement placed on:

pandas/core/groupby/generic.py
pandas/core/groupby/groupby.py
pandas/core/groupby/grouper.py
pandas/core/groupby/ops.py

ref #29701

ShaharNaveh · 2019-11-19T14:31:34Z

ohad83 · 2019-11-21T06:39:03Z

I'll take

pandas/plotting/_core.py
pandas/plotting/_matplotlib/boxplot.py
pandas/plotting/_matplotlib/converter.py
pandas/plotting/_matplotlib/core.py
pandas/plotting/_matplotlib/misc.py
pandas/plotting/_matplotlib/style.py
pandas/plotting/_matplotlib/timeseries.py
pandas/plotting/_matplotlib/tools.py
pandas/plotting/_misc.py

ref #29781

alimcmaster1 · 2019-11-24T21:27:33Z

I'll take:

pandas/core/reshape/concat.py

pandas/core/reshape/melt.py

pandas/core/reshape/merge.py

pandas/core/reshape/pivot.py

pandas/core/reshape/reshape.py

pandas/core/reshape/tile.py

What are everyone's thoughts on tagging this as a good first issue? It should apply to most of the files here. The changes that need to be made are usually only a few lines or so per file, and whoever is making the changes doesn't need to worry too much about affecting other parts of the code (since the end function performed is the same).

I'm picturing a setup similar to #28926.

Sure ive labelled accordingly. thanks

pandas-dev#34094)

… 2 files were fixed

OlivierLuG · 2020-05-25T20:35:02Z

The two files were modified:
pandas/_libs/tslibs/timedeltas.pyx
pandas/_libs/tslibs/timestamps.pyx

Note that there was no issues in the following ones. You can mark as done too:
pandas/_libs/tslibs/c_timestamp.pyx
pandas/_libs/tslibs/frequencies.pyx
pandas/_libs/tslibs/parsing.pyx
pandas/_libs/tslibs/period.pyx
pandas/_libs/tslibs/strptime.pyx

note: this is my first PR ever. Let me know if something need to be improved.

…es w… (#34370)

OlivierLuG · 2020-05-26T16:12:12Z

I've went through the topic to update the list + check some files.

Files marked as done without any commit

(no need to change anything):

Remaining files to check:

matteosantama · 2020-05-26T22:11:43Z

warden706 · 2020-05-26T22:15:32Z

Hi Matteo, I'm interested in helping with this effort, but am new to git and contributing to pandas. Would you possibly be able to walk me through the steps ? Perhaps I can setup a screen share this week? Thanks, Andrew

…

On Tue, May 26, 2020, 3:12 PM Matteo Santamaria ***@***.***> wrote: I took care of pandas/util/_validators.py. Many of these other files already seem ok to me too. - pandas/util/_test_decorators.py - pandas/tseries/frequencies.py - pandas/tests/util/test_assert_frame_equal.py - pandas/tests/tslibs/test_parsing.py - pandas/tests/tseries/holiday/test_holiday.py - pandas/tests/tseries/holiday/test_calendar.py - pandas/tests/tools/test_to_datetime.py - pandas/tests/test_strings.py - pandas/tests/series/test_repr.py - pandas/tests/series/test_datetime_values.py - pandas/tests/series/test_constructors.py - pandas/tests/series/test_api.py — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#29547 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APG73XKAUDLFVVQXUS2SJETRTQ5DDANCNFSM4JLZCU5Q> .

matteosantama · 2020-05-26T22:20:33Z

Hey @warden706, I'm actually pretty new here too, so I wouldn't have much to show you. I've found this resource very helpful as I've stumbled around, you should check it out.

MatteoFelici · 2020-05-27T06:37:33Z

DanBasson · 2020-05-28T14:31:13Z

i'm also new here.
i'll take

pandas/tests/series/indexing/test_numeric.py
pandas/tests/series/indexing/test_take.py
pandas/tests/series/indexing/test_where.py

i have a question regarding the code change.
for instance, in pandas/tests/series/indexing/test_take.py, snippet of the code:

 msg = "index {} is out of bounds for( axis 0 with)? size 5"
 with pytest.raises(IndexError, match=msg.format(10)):
     ser.take([1, 10])

so my suggestion is to replace it to:

msg = lambda x: f"index {x} is out of bounds for( axis 0 with)? size 5"
with pytest.raises(IndexError, match=msg(10)):
    ser.take([1, 10])

is that good enough?

MatteoFelici · 2020-05-28T21:46:39Z

Hi,
I'd like to make a PR, so I'm running the test, but I'm having a couple of fails. So I tried to run tests also on master.
Is it normal that running pytest pandas on unedited forked master returns a couple of fails?

matteosantama · 2020-05-28T21:56:08Z

Master should generally pass the tests. Make sure you've pulled the latest commits. Which tests are failing?

MatteoFelici · 2020-05-29T09:31:10Z

@matteosantama I pulled last commits, re-installed the environment an re-run the tests with pytest pandas. These are the results


================= short test summary info =================
FAILED pandas/tests/io/test_parquet.py::TestParquetFastParquet::test_s3_roundtrip - ValueError: Invalid timestamp "Ven, 29 Mag 2020 07:59:19 GMT": Unknown string format: Ven, 29 Mag 2020 07:59:19 GMT
FAILED pandas/tests/plotting/test_datetimelike.py::TestTSPlot::test_ts_plot_with_tz['UTC'] - AttributeError: 'numpy.datetime64' object has no attribute 'hour'
================= 2 failed, 87804 passed, 1185 skipped, 1005 xfailed, 5637 warnings in 2437.06s (0:40:37) =================

I noticed that if I run tests only on the single directory (for example with pytest pandas/tests/io), there are no fails:

 7273 passed, 344 skipped, 53 xfailed, 5584 warnings in 351.76s (0:05:51)

MatteoFelici · 2020-06-20T09:03:06Z

DanBasson · 2020-06-20T11:11:12Z

i keep getting errors which i don't know what they mean.
any help will be appreciated

MatteoFelici · 2020-06-20T11:47:03Z

Have you tried to fetch the latest modifications on master? Maybe it will fix some of the failed tests.

DanBasson · 2020-06-20T12:21:24Z

it didn't help.
if someone else wants to take it, you can

MatteoFelici · 2020-06-20T16:56:05Z

I have a doubt: when we have a situation like in pandas/tests/reshape/test_melt.py:

msg = "The following '{Var}' are not present in the DataFrame: {Col}"
...
with pytest.raises(KeyError, match=msg.format(Var="value_vars", Col="\\['C'\\]")):
...
with pytest.raises(KeyError, match=msg.format(Var="id_vars", Col="\\['A'\\]")):
...

and so on, should we transform msg to a function and call it with different values of "Col"? Or is it better to leave it as it is?

WillAyd · 2020-06-26T15:28:40Z

@MatteoFelici thanks for that updated list. I checked the last few remaining modules you called out and this looks OK, so I think we can close this issue

ShaharNaveh changed the title ~~Go from old string formating to new F-string formatting.~~ Go from old string formating(% or .format()) to new F-string formatting. Nov 11, 2019

mroeschke mentioned this issue Nov 11, 2019

CLN: replace %s formatting syntax with .format #16130

Closed

ShaharNaveh changed the title ~~Go from old string formating(% or .format()) to new F-string formatting.~~ Replace old string formatting syntax with f-string format Nov 11, 2019

ShaharNaveh changed the title ~~Replace old string formatting syntax with f-string format~~ Replace old string formatting syntax with f-string Nov 11, 2019

ShaharNaveh mentioned this issue Nov 11, 2019

CLN:F-string formatting #29554

Merged

3 tasks

alimcmaster1 mentioned this issue Nov 11, 2019

CLN: String formatting % -> f-strings #29518

Merged

2 tasks

jbrockmendel added Clean Code Style Code style, linting, code_checks labels Nov 12, 2019

ShaharNaveh mentioned this issue Nov 13, 2019

CLN: f-string at pandas/_libs/tslib.pyx #29593

Merged

3 tasks

This was referenced Nov 13, 2019

CLN: Update .format(...) strings to f-expressions #29571

Merged

CLN: F-string formatting in pandas/tests/indexes/*.py (#29547) #29579

Merged

jreback pushed a commit that referenced this issue Nov 14, 2019

CLN: F-string formatting in pandas/tests/indexes/*.py (#29547) (#29579)

d1173de

This was referenced Nov 16, 2019

CLN:f-string asv #29663

Merged

CLN:F-strings #29662

Merged

Reksbril pushed a commit to Reksbril/pandas that referenced this issue Nov 18, 2019

CLN: F-string formatting in pandas/tests/indexes/*.py (pandas-dev#29547…

373263a

…) (pandas-dev#29579)

lucassa3 mentioned this issue Nov 19, 2019

format replaced with f-strings #29701

Merged

3 tasks

ShaharNaveh mentioned this issue Nov 21, 2019

CLN:F-string in pandas/_libs/tslibs/*.pyx #29775

Merged

5 tasks

ohad83 mentioned this issue Nov 21, 2019

CLN - Change string formatting in plotting #29781

Merged

3 tasks

alimcmaster1 added the good first issue label Nov 24, 2019

ForTimeBeing mentioned this issue Nov 26, 2019

Changed string formatting to fstring #29865

Closed

4 tasks

rhshadrach pushed a commit to rhshadrach/pandas that referenced this issue May 10, 2020

Replace old string formatting syntax with f-strings (pandas-dev#29547) (

a3ae0fe

pandas-dev#34094)

OlivierLuG added a commit to OlivierLuG/pandas that referenced this issue May 25, 2020

Replace old string formatting syntax with f-strings pandas-dev#29547:…

24643ee

… 2 files were fixed

OlivierLuG mentioned this issue May 25, 2020

CLN Replace old string formatting syntax with f-strings #29547: 2 files w… #34370

Merged

jreback pushed a commit that referenced this issue May 26, 2020

CLN Replace old string formatting syntax with f-strings #29547: 2 fil…

75c0e24

…es w… (#34370)

matteosantama mentioned this issue May 26, 2020

CLN: GH29547 Replace old string formatting with f-strings #34397

Closed

matteosantama mentioned this issue May 27, 2020

CLN: GH29547 format with f-strings #34399

Merged

MatteoFelici mentioned this issue Jun 16, 2020

CLN: GH29547 change string formatting with f-strings (6 files changed) #34831

Merged

2 tasks

This was referenced Jun 23, 2020

CLN: GH29547 format with f-strings #34914

Closed

CLN: GH29547 format with f-strings #35013

Closed

WillAyd closed this as completed Jun 26, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace old string formatting syntax with f-strings #29547

Replace old string formatting syntax with f-strings #29547

ShaharNaveh commented Nov 11, 2019 •

edited

Loading

ShaharNaveh commented Nov 11, 2019 •

edited

Loading

yashukla commented Nov 12, 2019 •

edited

Loading

SaturnFromTitan commented Nov 12, 2019 •

edited

Loading

ShaharNaveh commented Nov 12, 2019 •

edited

Loading

ShaharNaveh commented Nov 14, 2019 •

edited

Loading

yashukla commented Nov 15, 2019 •

edited

Loading

lucassa3 commented Nov 19, 2019 •

edited

Loading

ShaharNaveh commented Nov 19, 2019 •

edited

Loading

ohad83 commented Nov 21, 2019 •

edited

Loading

alimcmaster1 commented Nov 24, 2019

OlivierLuG commented May 25, 2020 •

edited

Loading

OlivierLuG commented May 26, 2020 •

edited

Loading

matteosantama commented May 26, 2020

warden706 commented May 26, 2020 via email

matteosantama commented May 26, 2020

MatteoFelici commented May 27, 2020 •

edited

Loading

DanBasson commented May 28, 2020 •

edited

Loading

MatteoFelici commented May 28, 2020

matteosantama commented May 28, 2020

MatteoFelici commented May 29, 2020

MatteoFelici commented Jun 20, 2020 •

edited by WillAyd

Loading

DanBasson commented Jun 20, 2020

MatteoFelici commented Jun 20, 2020

DanBasson commented Jun 20, 2020

MatteoFelici commented Jun 20, 2020

WillAyd commented Jun 26, 2020

Replace old string formatting syntax with f-strings #29547

Replace old string formatting syntax with f-strings #29547

Comments

ShaharNaveh commented Nov 11, 2019 • edited Loading

Notes:

Tip:

The current list is:

NOTE:

ShaharNaveh commented Nov 11, 2019 • edited Loading

yashukla commented Nov 12, 2019 • edited Loading

SaturnFromTitan commented Nov 12, 2019 • edited Loading

ShaharNaveh commented Nov 12, 2019 • edited Loading

ShaharNaveh commented Nov 14, 2019 • edited Loading

yashukla commented Nov 15, 2019 • edited Loading

lucassa3 commented Nov 19, 2019 • edited Loading

ShaharNaveh commented Nov 19, 2019 • edited Loading

ohad83 commented Nov 21, 2019 • edited Loading

alimcmaster1 commented Nov 24, 2019

OlivierLuG commented May 25, 2020 • edited Loading

OlivierLuG commented May 26, 2020 • edited Loading

Files marked as done without any commit

Remaining files to check:

matteosantama commented May 26, 2020

warden706 commented May 26, 2020 via email

matteosantama commented May 26, 2020

MatteoFelici commented May 27, 2020 • edited Loading

DanBasson commented May 28, 2020 • edited Loading

MatteoFelici commented May 28, 2020

matteosantama commented May 28, 2020

MatteoFelici commented May 29, 2020

MatteoFelici commented Jun 20, 2020 • edited by WillAyd Loading

Corrected

No need to modification

Still to check/correct

DanBasson commented Jun 20, 2020

MatteoFelici commented Jun 20, 2020

DanBasson commented Jun 20, 2020

MatteoFelici commented Jun 20, 2020

WillAyd commented Jun 26, 2020

ShaharNaveh commented Nov 11, 2019 •

edited

Loading

ShaharNaveh commented Nov 11, 2019 •

edited

Loading

yashukla commented Nov 12, 2019 •

edited

Loading

SaturnFromTitan commented Nov 12, 2019 •

edited

Loading

ShaharNaveh commented Nov 12, 2019 •

edited

Loading

ShaharNaveh commented Nov 14, 2019 •

edited

Loading

yashukla commented Nov 15, 2019 •

edited

Loading

lucassa3 commented Nov 19, 2019 •

edited

Loading

ShaharNaveh commented Nov 19, 2019 •

edited

Loading

ohad83 commented Nov 21, 2019 •

edited

Loading

OlivierLuG commented May 25, 2020 •

edited

Loading

OlivierLuG commented May 26, 2020 •

edited

Loading

MatteoFelici commented May 27, 2020 •

edited

Loading

DanBasson commented May 28, 2020 •

edited

Loading

MatteoFelici commented Jun 20, 2020 •

edited by WillAyd

Loading