Python bindings: implement __arrow_c_stream__() interface for ogr.Layer

[rouault]

@kylebarron Just seeing your yesterday toot https://mapstodon.space/@kylebarron/111723196242600250 , I gave this a try. I don't necessarily have a way to test that beyond the simple unit tests I added

[coveralls]

coverage: 68.739% (+0.003%) from 68.736%
when pulling e4f78be on rouault:python_arrow_c_stream
into 15751e3 on OSGeo:master.

[kylebarron]

I think requested_schema was added for future-proofing, but no one implements it yet. So it's fine to ignore

[kylebarron]

This is awesome!

The pyarrow.table constructor will check for __arrow_c_stream__ and call on input as of pyarrow v14, so I think that's the easiest way to test that it works end to end.

cc @jorisvandenbossche @paleolimbot

[rouault]

The pyarrow.table constructor will check for arrow_c_stream

Confirmed:

$ python3
>>> from osgeo import gdal
>>> import pyarrow
>>> ds = gdal.OpenEx("poly.shp")
>>> lyr = ds.GetLayer(0)
>>> table = pyarrow.table(lyr)
>>> table
pyarrow.Table
OGC_FID: int64 not null
AREA: double
EAS_ID: int64
PRFEDEA: string
wkb_geometry: binary
----
OGC_FID: [[0,1,2,3,4,5,6,7,8,9]]
AREA: [[215229.266,247328.172,261752.781,547597.188,15775.758,101429.977,268597.625,1634833.375,-596610.313,5268.813]]
EAS_ID: [[168,179,171,173,172,169,166,158,165,170]]
PRFEDEA: [["35043411","35043423","35043414","35043416","35043415","35043412","35043409","35043369","35043408","35043413"]]
wkb_geometry: [[01030000000100000014000000000000602F491D41000000207F2D5241000000C028471D41000000E0922D5241000000007C461D4100000060AE2D524100000080C9471D4100000020B62D5241000000209C4C1D41000000E0D82D5241000000608D4C1D41000000A0DD2D5241000000207F4E1D41000000A0EA2D524100000020294F1D4100000080CA2D524100000000B4511D41000000E0552D5241000000C016521D4100000080452D5241000000E0174E1D41000000202E2D524100000020414D1D41000000E04C2D5241000000E04B4D1D41000000605E2D524100000040634D1D41000000E0742D5241000000A0EF4C1D41000000E08D2D5241000000E04E4C1D41000000E0A12D5241000000E0B04B1D4100000060B82D524100000080974A1D4100000080AE2D524100000080CF491D4100000080952D5241000000602F491D41000000207F2D5241,01030000000100000014000000000000608D4C1D41000000A0DD2D5241000000209C4C1D41000000E0D82D524100000080C9471D4100000020B62D5241000000007C461D4100000060AE2D5241000000C028471D41000000E0922D5241000000602F491D41000000207F2D5241000000608F481D41000000606D2D52410000002007471D4100000000432D524100000000B0431D41000000A0432D5241000000C0CD421D4100000020432D524100000000E4411D41000000E0422D5241000000A0A23F1D41000000A0342D524100000040733F1D41000000C0332D5241000000204D3E1D41000000C0552D5241000000201A3D1D41000000406F2D5241000000E0D63C1D41000000A06D2D5241000000C09B3C1D41000000E0762D5241000000C0B73E1D4100000020842D5241000000805E461D41000000E0B52D5241000000608D4C1D41000000A0DD2D5241,0103000000010000001D000000000000602F491D41000000207F2D524100000080CF491D4100000080952D524100000080974A1D4100000080AE2D5241000000E0B04B1D4100000060B82D5241000000E04E4C1D41000000E0A12D5241000000A0EF4C1D41000000E08D2D524100000040634D1D41000000E0742D5241000000E04B4D1D41000000605E2D524100000020014C1D41000000C05C2D5241000000E0FB4A1D4100000000572D5241000000805C4A1D4100000000522D524100000080D9471D4100000040302D52410000002001471D41000000002D2D52410000002073461D41000000E0262D52410000008057461D41000000C0202D5241000000E004461D4100000000162D5241000000208D451D41000000800B2D524100000020F0441D4100000060052D5241000000A043441D4100000060F02C5241000000401F401D4100000040D62C524100000040373E1D41000000C02B2D524100000040733F1D41000000C0332D5241000000A0A23F1D41000000A0342D524100000000E4411D41000000E0422D5241000000C0CD421D4100000020432D524100000000B0431D41000000A0432D52410000002007471D4100000000432D5241000000608F481D41000000606D2D5241000000602F491D41000000207F2D5241,0103000000010000001A000000000000C09B3C1D41000000E0762D5241000000E0D63C1D41000000A06D2D524100000040373E1D41000000C02B2D5241000000401F401D4100000040D62C52410000008027411D4100000040B22C52410000002060401D41000000A0A62C5241000000A0D33F1D41000000208D2C5241000000002A3E1D4100000080832C524100000020A93C1D41000000A0C02C5241000000C0C33A1D41000000C0B42C52410000004073381D4100000020A52C5241000000209C341D41000000808E2C524100000000F7331D4100000060AC2C5241000000C0AF331D4100000020BC2C524100000040BF331D4100000080C92C5241000000C0C2331D41000000E0DC2C5241000000C098341D4100000060DF2C5241000000006C371D41000000A00E2D52410000002074361D41000000201D2D52410000006015361D41000000A0172D52410000000012351D41000000E0052D524100000080F0331D41000000C0F12C52410000004020331D4100000080FB2C524100000020AE311D4100000000332D524100000000A53A1D41000000006B2D5241000000C09B3C1D41000000E0762D5241,01030000000100000006000000000000E0D63C1D41000000A06D2D5241000000201A3D1D41000000406F2D5241000000204D3E1D41000000C0552D524100000040733F1D41000000C0332D524100000040373E1D41000000C02B2D5241000000E0D63C1D41000000A06D2D5241,01030000000100000014000000000000E04B4D1D41000000605E2D524100000020414D1D41000000E04C2D5241000000E0174E1D41000000202E2D5241000000E0814B1D41000000001D2D5241000000C01A481D4100000080072D5241000000A0DF471D4100000000142D52410000006060461D41000000400C2D524100000060AA461D4100000080FF2C5241000000A043441D4100000060F02C524100000020F0441D4100000060052D5241000000208D451D41000000800B2D5241000000E004461D4100000000162D52410000008057461D41000000C0202D52410000002073461D41000000E0262D52410000002001471D41000000002D2D524100000080D9471D4100000040302D5241000000805C4A1D4100000000522D5241000000E0FB4A1D4100000000572D524100000020014C1D41000000C05C2D5241000000E04B4D1D41000000605E2D5241,0103000000010000001F000000000000C016521D4100000080452D5241000000A064541D4100000080552D5241000000E0DF541D41000000803D2D52410000000074551D41000000C0282D524100000040F4561D4100000080102D52410000000019571D41000000A0042D52410000006032551D4100000020112D5241000000E072541D4100000040112D5241000000800C541D41000000C0052D5241000000A06E571D4100000060E42C5241000000C0D0561D4100000060C92C52410000006097561D41000000C0C52C5241000000603D561D4100000060BB2C52410000008021551D4100000040B02C5241000000A0D1531D41000000A0A92C5241000000209A511D41000000409D2C524100000040014F1D41000000C0852C524100000080744E1D41000000A09A2C524100000040A94F1D4100000000A42C52410000008084501D4100000020AF2C524100000000F0501D41000000C0BA2C5241000000405E511D4100000040CF2C5241000000E05E511D4100000020DD2C52410000002008501D41000000C0172D5241000000A0C44E1D4100000080112D524100000000774D1D41000000C0072D5241000000002C4C1D41000000A0002D524100000040C44B1D41000000000F2D5241000000E0814B1D41000000001D2D5241000000E0174E1D41000000202E2D5241000000C016521D4100000080452D5241,0103000000010000003D00000000000040F4561D4100000080102D5241000000E0E5571D41000000801A2D5241000000E0E3581D4100000000252D524100000020685A1D41000000A02C2D5241000000E08A5A1D41000000002D2D524100000000615B1D41000000C0322D5241000000C0005D1D4100000000422D524100000060C35D1D41000000A0502D52410000004005601D4100000020332D5241000000602C601D4100000080222D5241000000A0E7621D4100000020322D524100000080C5621D4100000040422D5241000000A096631D41000000C0492D5241000000A00B641D41000000A04D2D5241000000009C641D41000000E0512D52410000008008651D41000000E03C2D52410000006077651D4100000060272D524100000040B5651D41000000601F2D5241000000E08F651D41000000E01E2D5241000000404D5F1D4100000080FA2C524100000040E75E1D4100000060F62C5241000000E0BC5E1D41000000E0F32C524100000000865E1D41000000C0EF2C5241000000C01F5E1D4100000000E92C5241000000E0D15D1D41000000A0D72C524100000080D65D1D4100000020D02C524100000060BF5E1D4100000000852C5241000000C0DD5D1D41000000A05A2C524100000040A95D1D41000000E0522C5241000000C03F5C1D41000000603C2C5241000000C0CA591D4100000080052C524100000040E8581D41000000A0002C5241000000C0A4571D4100000020FB2B5241000000C039561D41000000E0FA2B5241000000405F551D4100000000032C5241000000C067501D4100000020EE2B524100000000764F1D4100000060E32B5241000000C01E4F1D4100000040DD2B5241000000600C4F1D4100000080C92B5241000000C046501D4100000060AA2B5241000000C0D6501D41000000E06D2B524100000060274F1D41000000A0542B524100000040BD491D4100000020402B524100000020A2491D4100000040442B524100000080C3471D4100000060942B5241000000C012441D4100000080322C524100000040F8441D41000000E0432C5241000000E0E04D1D4100000020872C524100000040014F1D41000000C0852C5241000000209A511D41000000409D2C5241000000A0D1531D41000000A0A92C52410000008021551D4100000040B02C5241000000603D561D4100000060BB2C52410000006097561D41000000C0C52C5241000000C0D0561D4100000060C92C5241000000A06E571D4100000060E42C5241000000800C541D41000000C0052D5241000000E072541D4100000040112D52410000006032551D4100000020112D52410000000019571D41000000A0042D524100000040F4561D4100000080102D5241,0103000000010000001B000000000000C01A481D4100000080072D5241000000E0814B1D41000000001D2D524100000040C44B1D41000000000F2D5241000000002C4C1D41000000A0002D524100000000774D1D41000000C0072D5241000000A0C44E1D4100000080112D52410000002008501D41000000C0172D5241000000E05E511D4100000020DD2C5241000000405E511D4100000040CF2C524100000000F0501D41000000C0BA2C52410000008084501D4100000020AF2C524100000040A94F1D4100000000A42C524100000080744E1D41000000A09A2C524100000040014F1D41000000C0852C5241000000E0E04D1D4100000020872C524100000040F8441D41000000E0432C5241000000C012441D4100000080322C524100000000FF431D4100000020362C5241000000004B431D4100000080552C52410000000030431D41000000C05D2C5241000000C09B421D4100000000712C524100000080D6411D4100000080912C52410000008027411D4100000040B22C5241000000401F401D4100000040D62C5241000000A043441D4100000060F02C524100000060AA461D4100000080FF2C5241000000C01A481D4100000080072D5241,01030000000100000005000000000000C01A481D4100000080072D524100000060AA461D4100000080FF2C52410000006060461D41000000400C2D5241000000A0DF471D4100000000142D5241000000C01A481D4100000080072D5241]]

[jorisvandenbossche]

Nice!