Import very slow

[skjerns]

I figured that the import of this library is very slow (500-1000ms on my system).
Given that the library probably less complex than e.g. numpy or similarly large packages: Do you think the import times can be optimized in some way?

[noviluni]

Hi @skjerns , could you confirm that this is happening with the last version? I can't reproduce it.

[noviluni]

On the other side, the import is a little slower than other libraries because it initializes some parser to be faster when executing it, so I think there isn't too much we can do.

[skjerns]

@noviluni it's faster now (280ms), but I've also upgraded my PC, so I can't really tell ;-)

using python: 100ms (kind of acceptable)
using ipython: 280ms (this is still slower than scipy or numpy)

Maybe you could implement some lazy loading to the parsers? Or are they all needed?

[noviluni]

I tried with the last version of dateparser (1.0.0) and it takes less time than pandas, so I think it shouldn't be considered an issue.

I will close it. Feel free to comment or reopen it if you thing there is something that should be improved 🙂

[skjerns]

Yes! Seems to be much faster now :) thanks alot!

[Natureshadow]

dateparser 1.1.8 takes around 600ms to import here.

[skjerns]

dateparser 1.1.8 takes around 600ms to import here.

cannot replicate: Elapsed context: 275 ms on v1.1.8

[Natureshadow]

cannot replicate: Elapsed context: 275 ms

So, your system is faster than mine. 275ms import time is still unacceptable (basically everything >10ms on an average system is a bug)

…

On 13 July 2023 10:47:59 CEST, Simon Kern ***@***.***> wrote: > dateparser 1.1.8 takes around 600ms to import here. cannot replicate: `Elapsed context: 275 ms` -- Reply to this email directly or view it on GitHub: #533 (comment) You are receiving this because you commented. Message ID: ***@***.***>

[beda42]

I have to agree that hundreds of milliseconds are a problem. For me the import is about 250 ms and about 200 ms from that is the generation of _search_regex_ignorecase and related intermediate values at the end of timezone_parser.py:

_search_regex_parts = []
_tz_offsets = list(build_tz_offsets(_search_regex_parts))
_search_regex = re.compile('|'.join(_search_regex_parts))
_search_regex_ignorecase = re.compile(
    '|'.join(_search_regex_parts), re.IGNORECASE)

(from that 140 ms is just _tz_offsets = list(build_tz_offsets(_search_regex_parts)), but that does not really matter).

From my point of view, it would be better not to precompute this list on import, but rather do so on first use of pop_tz_offset_from_string which is the only function using it. That way the import time would be significantly shortened and the overall running time of an app using the pop_tz_offset_from_string function won't be affected.

Would it be interesting for you if I attempted this change and sent a pull request?

[beda42]

OK, I created an MR for this issue. It postpones the compilation of regexps in timezone_parser.py and reduces the import time to about 20 % of the original import time.

Any feedback is welcome.