[Python] Allow setting path to timezone db through python API

[rohanjain101]

Describe the enhancement requested

The timezone DB is required to be at %USERPROFILE%\Downloads\tzdata in pyarrow on windows. It would be nice to have a python API to change the path where this db should be found, I believe this is currently only possible through the C++ implementation using arrow:initialize.

It would also be nice if the timezone db was not required if tzdata/pytz are installed.

Component(s)

Python

[jorisvandenbossche]

It would also be nice if the timezone db was not required if tzdata/pytz are installed.

If I remember correctly, the problem here is that this uses a different format than what the tz implementation we use (https://howardhinnant.github.io/date/tz.html) expects.
See eg #28868 (comment) and #31472 for previous discussion about this.

[jorisvandenbossche]

And the R bindings already expose this, which essentially is a small wrapper around the setting a C++ option (arrow::GlobalOptions.timezone_db_path):

arrow/r/src/config.cpp

Lines 38 to 47 in f6e4479

	void set_timezone_database(cpp11::strings path) {
	auto paths = cpp11::as_cpp<std::vector<std::string>>(path);
	if (path.size() != 1) {
	cpp11::stop("Must provide a single path to the timezone database.");
	}
	arrow::GlobalOptions options;
	options.timezone_db_path = std::make_optional(paths[0]);
	arrow::StopIfNotOk(arrow::Initialize(options));
	}

So it should certainly be possible to add a similar function to pyarrow.

[mattijn]

Thanks for all the work on arrow!

I'm on windows and when running the following:

from datetime import datetime
import pyarrow.compute as pc

pc.strftime(datetime(2004, 8, 1))

I get the following error:

ArrowInvalid: Cannot locate timezone 'UTC': Timezone database not found at "D:\Users\Hoek\Downloads\tzdata"

Observe, that the error path refers to D:.

But in OP the following is mentioned:

The timezone DB is required to be at %USERPROFILE%\Downloads\tzdata in pyarrow on windows.

But if I check:

! echo %USERPROFILE%

C:\Users\Hoek

and

from pathlib import Path
Path.home()

WindowsPath('C:/Users/Hoek')

It is both linking to C:. How you know it is on D:? D: is right though..
There is no Downloads folder for me on C:\Users\Hoek (therefor https://stackoverflow.com/a/74292266/2459096 errored)

I assume it is just early days and this user experience will improve soon.
Again, thanks for all the hard work!

[jorisvandenbossche]

@mattijn what does import os; os.path.expanduser("~/Downloads") return for you?

[mattijn]

'C:\\Users\\Hoek/Downloads'

Apparently my Downloads location is not on C. See Downloads dialog:

[jorisvandenbossche]

So then the error message could be expected? On the C++ side, it is using wordexp for the path expansion of ~\Downloads, no idea how that exactly behaves on Windows.

[mattijn]

Yes. The error message is right indeed, C++ gets the right location using wordexp.
Using %USERPROFILE% or os.path.expanduser("~/Downloads") not.