Warning: Pandas Concat Empty DataFrame

[apalacio9502]

Hello,

We are encountering this warning with BDH when some of the queried tickers have no information, how can we suppress it.

FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.

Regards,

[avantgardeam]

Hello,

Could you provide a minimal reproducible example of the query causing the error?

[mmtpo]

I've been enjoying using your wrapper for blpapi. Recently, I encountered the same FutureWarning when using bdh function. Here is a min reprex:

Environment

• Python version: 3.10.9
• blp version: 0.0.3
• blpapi version: 3.23.1
• pandas version: 2.1.4

Reproducible Example

from blp import blp
bquery = blp.BlpQuery().start()

bquery.bdh(
    ["CZTA3MAY Index", "GTCZK2Y Govt"],
    ["PX_LAST"],
    start_date="20240208",
    end_date="20240208"
)

[slmg]

Hi, based on this SO thread, the below should silence this warning.

blp/src/blp/blp.py

Line 775 in 2369b8e

pandas.concat(dfs)

changed to

pandas.concat([df for df in dfs if not df.empty]) 

[matthewgilbert]

Happy to take a PR on this.

Can someone post what the actual response from @mmtpo looks like? Ideally raw blpapi output and the parsed dataframe. Don't currently have access to a bbg connection.

[mmtpo]

Sure! The problem should be as mentioned by @simg, i.e. the first response returns empty data, hence resulting in an DataFrame with no rows causing the warning. See partial dumps of the responses for both securities and the given date as above:

'HistoricalDataResponse':
{'securityData': {'security': 'CZTA3MAY Index', 'eidData': [...], 'sequenceNumber': 0, 'fieldExceptions': [...], 'fieldData': [...]}}
{'security': 'CZTA3MAY Index', 'eidData': [], 'sequenceNumber': 0, 'fieldExceptions': [], 'fieldData': []}
{'fieldData':[]}
...

'HistoricalDataResponse':
{'securityData': {'security': 'GTCZK2Y Govt', 'eidData': [...], 'sequenceNumber': 1, 'fieldExceptions': [...], 'fieldData': [...]}}
{'security': 'GTCZK2Y Govt', 'eidData': [], 'sequenceNumber': 1, 'fieldExceptions': [], 'fieldData': [{...}]}
{'fieldData': {'date': Timestamp('2024-02-0...00:00:00'), 'PX_LAST': 3.828}}

[simg]

The problem should be as mentioned by @simg @slmg

FTFY :)