[Bug]: embeddings_manager: required argument is not a float

[T3chArmy]

Checklist

• I have updated to the latest available Frigate version.
• I have cleared the cache of my browser.
• I have tried a different browser to see if it is related to my browser.
• I have tried reproducing the issue in incognito mode to rule out problems with any third party extensions or plugins I have installed.

Describe the problem you are having

Process embeddings_manager:
2024-10-15 14:50:36.887623-05:002024-10-15 19:50:36.887239175  Traceback (most recent call last):
2024-10-15 14:50:36.887631-05:002024-10-15 19:50:36.887240493    File "/opt/frigate/frigate/util/builtin.py", line 360, in serialize
2024-10-15 14:50:36.887649-05:002024-10-15 19:50:36.887241717      return struct.pack("%sf" % len(vector), *vector)
2024-10-15 14:50:36.887656-05:002024-10-15 19:50:36.887242947  struct.error: required argument is not a float
2024-10-15 14:50:36.887662-05:002024-10-15 19:50:36.887243914  
2024-10-15 14:50:36.887668-05:002024-10-15 19:50:36.887245120  During handling of the above exception, another exception occurred:
2024-10-15 14:50:36.887679-05:002024-10-15 19:50:36.887246026  
2024-10-15 14:50:36.887685-05:002024-10-15 19:50:36.887247029  Traceback (most recent call last):
2024-10-15 14:50:36.887691-05:002024-10-15 19:50:36.887248332    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-10-15 14:50:36.887697-05:002024-10-15 19:50:36.887249340      self.run()
2024-10-15 14:50:36.887703-05:002024-10-15 19:50:36.887250546    File "/opt/frigate/frigate/util/process.py", line 41, in run_wrapper
2024-10-15 14:50:36.887713-05:002024-10-15 19:50:36.887251586      return run(*args, **kwargs)
2024-10-15 14:50:36.887719-05:002024-10-15 19:50:36.887252891    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-10-15 14:50:36.887725-05:002024-10-15 19:50:36.887254016      self._target(*self._args, **self._kwargs)
2024-10-15 14:50:36.887731-05:002024-10-15 19:50:36.887255954    File "/opt/frigate/frigate/embeddings/__init__.py", line 59, in manage_embeddings
2024-10-15 14:50:36.887742-05:002024-10-15 19:50:36.887257008      maintainer = EmbeddingMaintainer(
2024-10-15 14:50:36.887749-05:002024-10-15 19:50:36.887276938    File "/opt/frigate/frigate/embeddings/maintainer.py", line 50, in __init__
2024-10-15 14:50:36.887754-05:002024-10-15 19:50:36.887278110      self.embeddings.reindex()
2024-10-15 14:50:36.887766-05:002024-10-15 19:50:36.887279392    File "/opt/frigate/frigate/embeddings/embeddings.py", line 267, in reindex
2024-10-15 14:50:36.887772-05:002024-10-15 19:50:36.887280588      self.batch_upsert_description(batch_descs)
2024-10-15 14:50:36.887778-05:002024-10-15 19:50:36.887281897    File "/opt/frigate/frigate/embeddings/embeddings.py", line 189, in batch_upsert_description
2024-10-15 14:50:36.887784-05:002024-10-15 19:50:36.887283014      items.append(serialize(embeddings[i]))
2024-10-15 14:50:36.887794-05:002024-10-15 19:50:36.887295061    File "/opt/frigate/frigate/util/builtin.py", line 364, in serialize
2024-10-15 14:50:36.887800-05:002024-10-15 19:50:36.887296315      raise ValueError(f"Failed to pack vector: {e}. Vector: {vector}")
2024-10-15 14:50:36.887806-05:002024-10-15 19:50:36.887297842  ValueError: Failed to pack vector: required argument is not a float. Vector: [array([-2.7319e-01,  1.5796e-01, -1.1945e-01,  7.7515e-02, -6.9031e-02,
2024-10-15 14:50:36.887816-05:002024-10-15 19:50:36.887299116          4.4385e-01,  2.2537e-02, -1.6220e-02,  1.1551e-02,  1.9312e-01,
2024-10-15 14:50:36.887822-05:002024-10-15 19:50:36.887300312          1.6528e-01,  5.1221e-01,  4.2065e-01, -4.4043e-01, -1.7395e-01,
2024-10-15 14:50:36.887829-05:002024-10-15 19:50:36.887301483         -7.7698e-02,  8.5010e-01, -1.0193e-01,  5.7227e-01, -2.6001e-01,
2024-10-15 14:50:36.887839-05:002024-10-15 19:50:36.887314908          6.1084e-01, -2.5055e-02,  8.3801e-02, -1.1346e-01,  3.6475e-01,
2024-10-15 14:50:36.887845-05:002024-10-15 19:50:36.887316122          2.7734e-01,  2.9688e-01,  5.1367e-01, -2.0459e-01,  1.0841e-02,
2024-10-15 14:50:36.887851-05:002024-10-15 19:50:36.887317374         -3.3051e-02, -2.7124e-01, -3.1281e-02, -5.4346e-01, -8.7646e-02,
2024-10-15 14:50:36.887857-05:002024-10-15 19:50:36.887318620         -2.9004e-01,  5.8203e-01, -7.1045e-02, -1.1890e-01, -5.0195e-01,
2024-10-15 14:50:36.887867-05:002024-10-15 19:50:36.887319976          1.7175e-01, -9.0820e-02,  4.2847e-01, -2.4036e-01, -5.2100e-01,
2024-10-15 14:50:36.887873-05:002024-10-15 19:50:36.887321172          3.4326e-01,  8.1665e-02, -1.5002e-01, -5.0476e-02, -2.7026e-01,
2024-10-15 14:50:36.887879-05:002024-10-15 19:50:36.887322310          8.2275e-02, -8.8501e-02, -6.5088e-01, -1.4076e-02,  1.8103e-01,
2024-10-15 14:50:36.887889-05:002024-10-15 19:50:36.887323478          3.9673e-01,  4.5874e-01,  4.5752e-01, -8.2092e-02,  7.5342e-01,
2024-10-15 14:50:36.887895-05:002024-10-15 19:50:36.887324780         -2.7295e-01,  1.0516e-01, -2.9272e-01,  2.1313e-01,  1.1646e-01,
2024-10-15 14:50:36.887901-05:002024-10-15 19:50:36.887325906          3.8428e-01,  1.6931e-01,  1.0144e-01,  6.2744e-02, -1.0669e-01,
2024-10-15 14:50:36.887908-05:002024-10-15 19:50:36.887327029          1.7444e-01, -1.0178e-02, -4.6558e-01,  1.3110e-01, -3.6548e-01,
2024-10-15 14:50:36.887918-05:002024-10-15 19:50:36.887328177          1.1597e-01, -1.5100e-01,  5.7666e-01,  8.8623e-02, -5.3320e-01,
2024-10-15 14:50:36.887924-05:002024-10-15 19:50:36.887329365         -2.7863e-02,  5.9863e-01, -5.1758e-01,  4.1284e-01, -5.7251e-02,
2024-10-15 14:50:36.887930-05:002024-10-15 19:50:36.887344112         -2.4023e-01,  2.7051e-01, -7.3193e-01,  2.9346e-01, -4.2653e-04,
2024-10-15 14:50:36.887942-05:002024-10-15 19:50:36.887345380          5.0476e-02,  5.3076e-01,  2.1094e-01, -6.4453e-02, -7.1045e-02,
2024-10-15 14:50:36.887953-05:002024-10-15 19:50:36.887346535          2.6099e-01, -2.7298e-02, -1.1359e-01, -1.7700e-01,  2.2583e-01,
2024-10-15 14:50:36.887960-05:002024-10-15 19:50:36.887347721         -3.2471e-02,  2.1680e-01, -2.2290e-01, -2.8320e-01,  2.7979e-01,
2024-10-15 14:50:36.887966-05:002024-10-15 19:50:36.887348875         -1.2683e-01, -1.2286e-01,  1.7139e-01,  3.9258e-01,  5.1086e-02,
2024-10-15 14:50:36.887977-05:002024-10-15 19:50:36.887350142         -1.9995e-01, -2.1130e-01, -4.8584e-01, -2.5684e-01, -1.2408e-01,
2024-10-15 14:50:36.887983-05:002024-10-15 19:50:36.887351307         -1.4198e-02, -3.2166e-02, -3.3960e-01, -1.1212e-01, -2.5098e-01,
2024-10-15 14:50:36.887989-05:002024-10-15 19:50:36.887352412          2.9160e-02,  8.5693e-01,  2.8638e-01, -7.7393e-01, -1.6467e-01,
2024-10-15 14:50:36.887999-05:002024-10-15 19:50:36.887353612          9.2529e-02,  1.6394e-01, -7.7209e-02,  7.4463e-01, -7.7454e-02,
2024-10-15 14:50:36.888005-05:002024-10-15 19:50:36.887354806          1.4258e-01, -2.1622e-02,  2.9395e-01,  8.4839e-02, -4.8584e-02,
2024-10-15 14:50:36.888011-05:002024-10-15 19:50:36.887355952          1.6418e-01,  2.0154e-01,  3.6060e-01,  1.9177e-01,  1.6956e-01,
2024-10-15 14:50:36.888017-05:002024-10-15 19:50:36.887386373          2.3889e-01,  1.7883e-01,  2.1350e-01, -5.0488e-01, -1.3831e-01,
2024-10-15 14:50:36.888026-05:002024-10-15 19:50:36.887387679         -1.9714e-01, -7.4036e-02, -1.1792e-01,  3.2788e-01, -2.2583e-01,
2024-10-15 14:50:36.888033-05:002024-10-15 19:50:36.887388871         -9.9121e-02,  5.7526e-02, -4.9316e-01,  2.1558e-01, -3.2300e-01,
2024-10-15 14:50:36.888038-05:002024-10-15 19:50:36.887390046         -6.0205e-01,  1.8616e-01, -4.2305e-03,  3.9795e-01,  4.1431e-01,
2024-10-15 14:50:36.888048-05:002024-10-15 19:50:36.887391177          2.2705e-02, -2.4609e-01, -4.0991e-01,  3.0533e-02, -2.2644e-02,
2024-10-15 14:50:36.888055-05:002024-10-15 19:50:36.887392321         -6.6992e-01, -2.7069e-02, -1.6357e-01,  1.6003e-01, -4.5679e-01,
2024-10-15 14:50:36.888061-05:002024-10-15 19:50:36.887393488         -3.9209e-01,  1.8518e-01,  1.2672e-02, -9.9243e-02,  3.7354e-02,
2024-10-15 14:50:36.888066-05:002024-10-15 19:50:36.887394887          2.8296e-01, -1.2451e+00,  8.5449e-02,  6.0516e-02, -5.3680e-02,
2024-10-15 14:50:36.888076-05:002024-10-15 19:50:36.887396055         -2.4744e-01, -4.5410e-02, -8.7814e-03, -1.8945e-01, -1.5030e-03,
2024-10-15 14:50:36.888082-05:002024-10-15 19:50:36.887397292          3.4082e-01,  5.0977e-01,  3.4619e-01, -4.7722e-03,  1.9421e-01,
2024-10-15 14:50:36.888088-05:002024-10-15 19:50:36.887398453         -4.6216e-01, -9.9548e-02,  1.2415e-01,  2.0154e-01,  8.1787e-02,
2024-10-15 14:50:36.888099-05:002024-10-15 19:50:36.887399652          5.9540e-02, -1.6382e-01,  5.3711e-01,  1.1761e-01,  1.5955e-01,
2024-10-15 14:50:36.888106-05:002024-10-15 19:50:36.887400846         -1.2295e+00,  8.0505e-02,  1.6235e-01, -4.1968e-01,  4.6826e-01,
2024-10-15 14:50:36.888111-05:002024-10-15 19:50:36.887401984         -1.8628e-01, -5.8203e-01,  3.0884e-01, -2.0294e-02, -3.1021e-02,
2024-10-15 14:50:36.888117-05:002024-10-15 19:50:36.887403120          1.2436e-02,  5.4834e-01, -2.0715e-01,  1.5820e-01, -5.5371e-01,
2024-10-15 14:50:36.888127-05:002024-10-15 19:50:36.887404338         -3.3813e-01, -1.7175e-01,  1.3342e-01, -5.6982e-01,  3.6621e-02,
2024-10-15 14:50:36.888134-05:002024-10-15 19:50:36.887405486          3.4814e-01, -1.3367e-01,  5.3613e-01,  3.4521e-01,  1.2952e-01,
2024-10-15 14:50:36.888141-05:002024-10-15 19:50:36.887406675         -2.3230e-01, -2.4072e-01,  1.6394e-01, -4.7638e-02, -2.6196e-01,
2024-10-15 14:50:36.888152-05:002024-10-15 19:50:36.887407847         -2.4548e-03,  5.1172e-01, -4.1473e-02, -6.0608e-02,  1.3782e-01,
2024-10-15 14:50:36.888158-05:002024-10-15 19:50:36.887409001         -4.4586e-02,  1.1682e-01, -4.4739e-02,  3.3008e-01, -2.7686e-01,
2024-10-15 14:50:36.888164-05:002024-10-15 19:50:36.887410142         -3.6719e-01, -4.5020e-01, -3.7817e-01,  9.8877e-02, -9.7418e-04,
2024-10-15 14:50:36.888176-05:002024-10-15 19:50:36.887411354          1.4307e-01, -1.9666e-01,  7.5732e-01, -1.6736e-01,  1.0022e-01,
2024-10-15 14:50:36.888182-05:002024-10-15 19:50:36.887412526         -3.7036e-01, -1.4355e-01,  2.9834e-01, -2.5574e-02,  6.1963e-01,
2024-10-15 14:50:36.888188-05:002024-10-15 19:50:36.887413689         -9.3750e-02, -2.1521e-01, -2.0667e-01, -1.1804e-01, -5.9766e-01,
2024-10-15 14:50:36.888194-05:002024-10-15 19:50:36.887434993         -3.5547e-01, -1.6931e-01,  1.0022e-01, -7.1924e-01, -6.9275e-02,
2024-10-15 14:50:36.888204-05:002024-10-15 19:50:36.887436229         -1.8542e-01, -5.5078e-01,  4.4214e-01,  3.4424e-01,  2.7588e-01,
2024-10-15 14:50:36.888210-05:002024-10-15 19:50:36.887437416         -3.8849e-02, -3.9746e-01,  7.3242e-01,  8.8684e-02, -1.7493e-01,
2024-10-15 14:50:36.888216-05:002024-10-15 19:50:36.887438551         -1.6418e-01,  2.8394e-01, -6.2598e-01, -1.5710e-01, -2.7954e-01,
2024-10-15 14:50:36.888226-05:002024-10-15 19:50:36.887439683         -2.7695e-02, -5.4883e-01,  1.7566e-01, -3.7817e-01,  4.7546e-02,
2024-10-15 14:50:36.888233-05:002024-10-15 19:50:36.887440817          1.1176e-01,  3.1055e-01, -2.6514e-01,  2.3767e-01,  9.8730e-01,
2024-10-15 14:50:36.888249-05:002024-10-15 19:50:36.887441974         -2.1716e-01,  3.7305e-01, -3.3984e-01,  1.1749e-02, -3.3716e-01,
2024-10-15 14:50:36.888267-05:002024-10-15 19:50:36.887443090          2.7710e-01,  1.0750e-02, -3.1372e-01, -4.0479e-01,  2.4707e-01,
2024-10-15 14:50:36.888284-05:002024-10-15 19:50:36.887444222         -2.4182e-01, -2.2754e-01,  4.9048e-01,  9.5276e-02,  2.9053e-01,
2024-10-15 14:50:36.888291-05:002024-10-15 19:50:36.887445361         -2.1729e-01, -1.8359e-01, -1.8909e-01, -7.4097e-02, -1.8848e-01,
2024-10-15 14:50:36.888297-05:002024-10-15 19:50:36.887464623          7.4890e-02, -1.8408e-01,  1.5732e-02, -9.3002e-03,  8.0713e-01,
2024-10-15 14:50:36.888309-05:002024-10-15 19:50:36.887465858          1.3281e-01, -2.4353e-01, -2.1509e-01,  9.1943e-01, -9.2346e-02,
2024-10-15 14:50:36.888316-05:002024-10-15 19:50:36.887467007         -1.1493e-01, -6.6284e-02,  7.1777e-02,  7.9541e-01,  6.5137e-01,
2024-10-15 14:50:36.888322-05:002024-10-15 19:50:36.887468132          2.1460e-01,  4.2529e-01,  1.5930e-01, -2.8882e-01,  5.7922e-02,
2024-10-15 14:50:36.888328-05:002024-10-15 19:50:36.887469313         -4.0015e-01, -5.3253e-02,  6.7529e-01, -3.0859e-01, -9.0515e-02,
2024-10-15 14:50:36.888338-05:002024-10-15 19:50:36.887470478         -1.4636e-01, -4.4434e-01, -2.8662e-01, -5.8174e-03,  2.1924e-01,
2024-10-15 14:50:36.888344-05:002024-10-15 19:50:36.887471627         -1.7242e-02, -4.1931e-02, -8.1604e-02,  4.9780e-01, -2.8052e-01,
2024-10-15 14:50:36.888350-05:002024-10-15 19:50:36.887472856          2.6611e-01, -3.0200e-01,  1.2122e-01, -2.1484e-01,  6.9092e-02,
2024-10-15 14:50:36.888361-05:002024-10-15 19:50:36.887473990          2.8271e-01, -5.9863e-01, -4.7852e-01, -7.1594e-02,  4.2773e-01,
2024-10-15 14:50:36.888367-05:002024-10-15 19:50:36.887475134         -6.7200e-02, -3.2373e-01, -2.0972e-01,  2.5903e-01, -2.9395e-01,
2024-10-15 14:50:36.888373-05:002024-10-15 19:50:36.887476310          6.6101e-02, -6.1035e-01, -3.9111e-01, -7.9407e-02,  4.1357e-01,
2024-10-15 14:50:36.888379-05:002024-10-15 19:50:36.887481233         -1.5114e-02, -4.5410e-01, -4.0479e-01, -2.4561e-01, -4.6899e-01,
2024-10-15 14:50:36.888390-05:002024-10-15 19:50:36.887482487         -7.9269e-03, -3.6792e-01, -2.6733e-01, -7.5073e-02, -2.2681e-01,
2024-10-15 14:50:36.888396-05:002024-10-15 19:50:36.887483693         -1.4978e-01,  4.4922e-01, -2.6929e-01,  2.8491e-01, -3.6102e-02,
2024-10-15 14:50:36.888402-05:002024-10-15 19:50:36.887484835         -3.1372e-01,  4.6753e-01,  8.7036e-02,  9.8572e-02,  3.6890e-01,
2024-10-15 14:50:36.888413-05:002024-10-15 19:50:36.887485977         -1.9824e-01, -2.7197e-01,  1.7136e-02, -2.3303e-01, -3.2275e-01,
2024-10-15 14:50:36.888419-05:002024-10-15 19:50:36.887487158          3.5181e-01, -6.0400e-01, -7.2002e-04,  2.9932e-01, -1.0156e+00,
2024-10-15 14:50:36.888425-05:002024-10-15 19:50:36.887488381         -3.2446e-01,  6.9922e-01,  1.2048e-01, -3.9014e-01, -9.5459e-02,
2024-10-15 14:50:36.888431-05:002024-10-15 19:50:36.887489526          4.3506e-01, -2.6221e-01, -4.1290e-02,  1.2781e-01, -3.2910e-01,
2024-10-15 14:50:36.888442-05:002024-10-15 19:50:36.887490694         -3.2910e-01,  9.0637e-02, -6.9043e-01, -2.7603e-02,  6.0742e-01,
2024-10-15 14:50:36.888448-05:002024-10-15 19:50:36.887491898         -5.1904e-01,  3.8403e-01,  2.3730e-01, -1.0242e-01, -2.0984e-01,
2024-10-15 14:50:36.888454-05:002024-10-15 19:50:36.887493110         -6.3293e-02,  2.6855e-01,  2.9639e-01, -6.9727e-01,  1.7380e-02,
2024-10-15 14:50:36.888464-05:002024-10-15 19:50:36.887494245          1.5735e-01,  1.7480e-01,  3.1909e-01,  5.5762e-01, -1.1530e-01,
2024-10-15 14:50:36.888471-05:002024-10-15 19:50:36.887516248          2.3438e-01,  1.4929e-01, -6.6113e-01, -4.1504e-01,  6.3965e-01,
2024-10-15 14:50:36.888477-05:002024-10-15 19:50:36.887517618         -3.1055e-01,  4.1992e-02,  5.1758e-01,  4.4849e-01, -1.7151e-02,
2024-10-15 14:50:36.888482-05:002024-10-15 19:50:36.887518804          3.5522e-01, -1.1871e-01, -5.0830e-01, -1.3196e-01,  7.1191e-01,
2024-10-15 14:50:36.888493-05:002024-10-15 19:50:36.887519955          2.5803e-02, -2.5513e-01, -1.5747e-01,  2.2107e-01,  4.2529e-01,
2024-10-15 14:50:36.888500-05:002024-10-15 19:50:36.887521123         -4.7144e-01,  1.3855e-01, -3.0640e-01,  1.3618e-02,  3.1616e-01,
2024-10-15 14:50:36.888506-05:002024-10-15 19:50:36.887522283         -6.0400e-01, -1.4478e-01, -2.3511e-01,  1.5930e-01,  6.2012e-01,
2024-10-15 14:50:36.888517-05:002024-10-15 19:50:36.887523503         -5.2734e-01,  3.7964e-01, -7.5500e-02,  7.0068e-02, -1.9189e-01,
2024-10-15 14:50:36.888523-05:002024-10-15 19:50:36.887524618          3.6987e-01, -8.8770e-01, -1.5771e-01, -2.9663e-01, -3.0151e-01,
2024-10-15 14:50:36.888529-05:002024-10-15 19:50:36.887525773          3.4717e-01, -8.2550e-03,  6.0693e-01,  2.3022e-01,  2.8687e-01,
2024-10-15 14:50:36.888544-05:002024-10-15 19:50:36.887545816          2.4976e-01, -2.9248e-01, -1.9202e-01, -6.8994e-01, -5.0830e-01,
2024-10-15 14:50:36.888552-05:002024-10-15 19:50:36.887547202         -2.4951e-01,  1.0046e-01,  2.2473e-01,  1.8896e-01,  1.9397e-01,
2024-10-15 14:50:36.888558-05:002024-10-15 19:50:36.887548430         -6.7444e-02,  1.9336e-01,  4.2578e-01,  2.6660e-01,  6.9287e-01,
2024-10-15 14:50:36.888564-05:002024-10-15 19:50:36.887549622          6.3037e-01, -3.1860e-02, -5.7568e-01, -1.4819e-01,  3.1421e-01,
2024-10-15 14:50:36.888575-05:002024-10-15 19:50:36.887550814         -4.9170e-01, -9.4788e-02,  1.2793e-01, -2.5436e-02, -4.5142e-01,
2024-10-15 14:50:36.888582-05:002024-10-15 19:50:36.887551959          1.4343e-01, -2.0654e-01, -2.1619e-01,  2.0020e-01,  1.3855e-01,
2024-10-15 14:50:36.888588-05:002024-10-15 19:50:36.887553098         -1.6138e-01,  3.2837e-01,  4.6045e-01, -6.5918e-02, -8.3618e-02,
2024-10-15 14:50:36.888600-05:002024-10-15 19:50:36.887554305         -4.1968e-01, -1.6553e-01, -3.6957e-02, -1.9055e-01,  5.0391e-01,
2024-10-15 14:50:36.888606-05:002024-10-15 19:50:36.887555471         -4.9097e-01,  3.9819e-01, -1.2311e-01, -2.3462e-01,  1.0535e-01,
2024-10-15 14:50:36.888612-05:002024-10-15 19:50:36.887556694          3.1641e-01, -5.8319e-02,  5.8632e-03, -1.1725e-01,  2.2949e-01,
2024-10-15 14:50:36.888618-05:002024-10-15 19:50:36.887557862         -5.8008e-01,  1.0199e-01, -7.2363e-01,  3.3228e-01,  2.4866e-01,
2024-10-15 14:50:36.888630-05:002024-10-15 19:50:36.887559092         -2.1741e-01, -2.3755e-01, -9.3155e-03, -7.9407e-02, -2.8589e-01,
2024-10-15 14:50:36.888636-05:002024-10-15 19:50:36.887560288         -1.6464e-02,  1.9482e-01, -1.7737e-01,  2.0361e-01, -3.7109e-01,
2024-10-15 14:50:36.888642-05:002024-10-15 19:50:36.887561523          4.1162e-01,  1.1169e-01, -4.1919e-01, -4.0186e-01,  5.2185e-03,
2024-10-15 14:50:36.888653-05:002024-10-15 19:50:36.887562717          1.8215e-03, -3.6768e-01,  5.1660e-01,  3.0786e-01, -1.8799e-01,
2024-10-15 14:50:36.888659-05:002024-10-15 19:50:36.887563870          8.4045e-02,  6.1096e-02,  8.4961e-02, -2.0117e-01, -1.0535e-01,
2024-10-15 14:50:36.888665-05:002024-10-15 19:50:36.887565045          3.2739e-01,  1.7529e-01, -1.8347e-01, -2.4719e-02, -2.3697e-02,
2024-10-15 14:50:36.888671-05:002024-10-15 19:50:36.887566260          7.1899e-02, -7.1436e-01,  2.8662e-01, -1.6663e-01, -4.3335e-02,
2024-10-15 14:50:36.888682-05:002024-10-15 19:50:36.887567411         -6.4551e-01,  4.2383e-01, -4.0344e-02, -6.8115e-01,  6.2402e-01,
2024-10-15 14:50:36.888688-05:002024-10-15 19:50:36.887568560          1.1169e-01, -1.9250e-01, -2.1826e-01,  3.0078e-01,  5.2832e-01,
2024-10-15 14:50:36.888694-05:002024-10-15 19:50:36.887569779         -8.3984e-02,  1.8347e-01,  5.8203e-01,  5.7568e-01, -3.3276e-01,
2024-10-15 14:50:36.888705-05:002024-10-15 19:50:36.887570937         -1.0559e-01, -3.4937e-01, -6.9043e-01,  7.9834e-02, -2.0935e-01,
2024-10-15 14:50:36.888711-05:002024-10-15 19:50:36.887572085          3.7109e-01, -5.3711e-02,  5.6519e-02,  3.1836e-01, -3.6682e-02,
2024-10-15 14:50:36.888717-05:002024-10-15 19:50:36.887573236          9.7534e-02, -3.4277e-01, -2.5195e-01, -8.6853e-02,  4.9243e-01,
2024-10-15 14:50:36.888723-05:002024-10-15 19:50:36.887574457         -8.4045e-02, -8.4167e-02,  2.8271e-01,  1.2695e-01,  3.6774e-02,
2024-10-15 14:50:36.888734-05:002024-10-15 19:50:36.887575623          5.3760e-01, -1.3232e-01,  4.7461e-01, -6.7676e-01,  7.3779e-01,
2024-10-15 14:50:36.888741-05:002024-10-15 19:50:36.887576815         -1.7136e-02,  8.5742e-01,  3.6914e-01, -3.9111e-01, -6.4880e-02,
2024-10-15 14:50:36.888747-05:002024-10-15 19:50:36.887577955          3.0347e-01, -1.8860e-01, -2.5684e-01, -1.3538e-01,  4.9854e-01,
2024-10-15 14:50:36.888757-05:002024-10-15 19:50:36.887579094          1.6772e-01,  2.1765e-01, -2.3193e-01,  4.4952e-02, -1.5442e-02,
2024-10-15 14:50:36.888764-05:002024-10-15 19:50:36.887580282         -4.2877e-02, -2.1094e-01,  2.6099e-01, -7.3486e-02,  3.0151e-01,
2024-10-15 14:50:36.888770-05:002024-10-15 19:50:36.887581410          2.4316e-01,  8.7509e-03,  2.5659e-01, -1.0236e-01, -5.1346e-03,
2024-10-15 14:50:36.888776-05:002024-10-15 19:50:36.887582571          6.0254e-01,  2.6352e-02, -1.4355e-01,  4.3408e-01, -3.8232e-01,
2024-10-15 14:50:36.888787-05:002024-10-15 19:50:36.887583750         -1.8335e-01, -4.8126e-02,  8.1641e-01, -4.5166e-01, -1.9849e-01,
2024-10-15 14:50:36.888794-05:002024-10-15 19:50:36.887584914         -6.7566e-02,  1.4990e-01,  9.0698e-02,  4.6356e-02,  1.9730e-02,
2024-10-15 14:50:36.888800-05:002024-10-15 19:50:36.887586149         -5.8380e-02, -3.9111e-01,  2.7100e-02,  2.1411e-01, -1.8616e-01,
2024-10-15 14:50:36.888811-05:002024-10-15 19:50:36.887587333          8.5022e-02,  4.3793e-02,  2.8882e-01, -2.0105e-01, -2.4561e-01,
2024-10-15 14:50:36.888817-05:002024-10-15 19:50:36.887588451         -3.2812e-01,  2.3389e-01,  4.9780e-01,  6.0059e-02,  4.7314e-01,
2024-10-15 14:50:36.888823-05:002024-10-15 19:50:36.887589632          3.7256e-01,  9.1113e-01, -1.7053e-01,  7.3926e-01,  2.9565e-01,
2024-10-15 14:50:36.888834-05:002024-10-15 19:50:36.887590759          4.9268e-01,  4.6875e-01, -5.7031e-01, -2.7435e-02, -9.7998e-01,
2024-10-15 14:50:36.888846-05:002024-10-15 19:50:36.887591901         -3.2690e-01, -1.3367e-01, -4.3433e-01, -9.1736e-02, -6.2988e-01,
2024-10-15 14:50:36.888853-05:002024-10-15 19:50:36.887593033          1.0419e-01,  1.2695e-01,  1.8079e-01,  2.0337e-01,  2.5732e-01,
2024-10-15 14:50:36.888859-05:002024-10-15 19:50:36.887594162          2.4512e-01,  2.7539e-01,  1.5039e-01, -4.1675e-01, -4.7241e-01,
2024-10-15 14:50:36.888870-05:002024-10-15 19:50:36.887595304          3.4570e-01, -6.9629e-01,  2.7954e-01, -1.1670e-01, -1.4087e-01,
2024-10-15 14:50:36.888877-05:002024-10-15 19:50:36.887596462          3.4570e-01,  4.5728e-01,  7.0618e-02, -9.6741e-02, -4.2786e-02,
2024-10-15 14:50:36.888883-05:002024-10-15 19:50:36.887597663          3.3252e-01,  5.6732e-02, -1.3418e+00,  6.0498e-01,  3.0371e-01,
2024-10-15 14:50:36.888894-05:002024-10-15 19:50:36.887598857          4.0088e-01, -3.0322e-01,  1.4380e-01,  6.8604e-02,  5.0430e-03,
2024-10-15 14:50:36.888900-05:002024-10-15 19:50:36.887600050          4.2798e-01,  2.7734e-01, -2.1765e-01,  2.2241e-01, -2.5208e-02,
2024-10-15 14:50:36.888906-05:002024-10-15 19:50:36.887601205          3.1738e-01, -7.1777e-02,  1.0809e-01,  5.3076e-01, -1.5793e-02,
2024-10-15 14:50:36.888912-05:002024-10-15 19:50:36.887602351         -3.2654e-02, -5.1074e-01, -1.8967e-02, -4.1064e-01, -5.1123e-01,
2024-10-15 14:50:36.888923-05:002024-10-15 19:50:36.887603547         -1.5356e-01,  1.4819e-01,  3.1372e-01,  1.5186e-01, -4.3945e-01,
2024-10-15 14:50:36.888930-05:002024-10-15 19:50:36.887604730         -4.9609e-01, -1.1993e-01,  6.7725e-01,  1.6589e-01,  4.1553e-01,
2024-10-15 14:50:36.888936-05:002024-10-15 19:50:36.887605897          3.0005e-01,  3.8721e-01, -1.0651e-01,  1.9666e-01,  4.0039e-01,
2024-10-15 14:50:36.888947-05:002024-10-15 19:50:36.887607052         -2.5000e-01,  2.2839e-01, -1.2115e-01,  1.6174e-01,  3.2153e-01,
2024-10-15 14:50:36.888954-05:002024-10-15 19:50:36.887608203          1.9592e-01, -6.0156e-01, -4.1431e-01,  2.3364e-01,  1.5979e-01,
2024-10-15 14:50:36.888960-05:002024-10-15 19:50:36.887609407          1.3965e-01, -6.6211e-01, -1.6235e-01,  3.5034e-01, -1.6418e-01,
2024-10-15 14:50:36.888966-05:002024-10-15 19:50:36.887610612         -1.1383e-02, -4.7046e-01,  1.7090e-01, -6.5723e-01,  5.6030e-02,
2024-10-15 14:50:36.888977-05:002024-10-15 19:50:36.887611732          2.1350e-01, -2.8198e-01, -2.4402e-01,  3.7280e-01, -3.9990e-01,
2024-10-15 14:50:36.888983-05:002024-10-15 19:50:36.887612843         -1.8835e-01, -1.3025e-01, -4.8096e-01,  1.4319e-01, -2.1875e-01,
2024-10-15 14:50:36.888989-05:002024-10-15 19:50:36.887613980          6.7432e-01, -1.0215e+00,  3.3350e-01, -1.9287e-01, -2.2803e-01,
2024-10-15 14:50:36.889000-05:002024-10-15 19:50:36.887615112          2.2278e-01, -3.6133e-01,  2.9614e-01], dtype=float16)]

Steps to reproduce

1. Enable Embeddings reindexing
2. Restart Frigate

Version

0.15.0-b75efcb

In which browser(s) are you experiencing the issue with?

No response

Frigate config file

# yaml-language-server: $schema=http://10.112.0.4:5000/api/config/schema.json

go2rtc:
  streams:
    front_yard:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.112.0.10/cam/realmonitor?channel=1&subtype=0
      - ffmpeg:front_yard#audio=opus
    back_yard:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.112.0.11/cam/realmonitor?channel=1&subtype=0
      - ffmpeg:back_yard#audio=opus
    rear_garage:
      - nest:?client_id={FRIGATE_NEST_CLIENT_ID}&client_secret={FRIGATE_NEST_CLIENT_SECRET}&device_id={FRIGATE_NEST_DEVICE_ID_REARGARAGE}&project_id={FRIGATE_NEST_PROJECT_ID}&refresh_token={FRIGATE_NEST_REFRESH_TOKEN}
    cat_room:
      - nest:?client_id={FRIGATE_NEST_CLIENT_ID}&client_secret={FRIGATE_NEST_CLIENT_SECRET}&device_id={FRIGATE_NEST_DEVICE_ID_CATROOM}&project_id={FRIGATE_NEST_PROJECT_ID}&refresh_token={FRIGATE_NEST_REFRESH_TOKEN}
    back_yard_sw:
      - nest:?client_id={FRIGATE_NEST_CLIENT_ID}&client_secret={FRIGATE_NEST_CLIENT_SECRET}&device_id={FRIGATE_NEST_DEVICE_ID_BACKYARD_SW}&project_id={FRIGATE_NEST_PROJECT_ID}&refresh_token={FRIGATE_NEST_REFRESH_TOKEN}#audio=opus
    front_door:
      - nest:?client_id={FRIGATE_NEST_CLIENT_ID}&client_secret={FRIGATE_NEST_CLIENT_SECRET}&device_id={FRIGATE_NEST_DEVICE_ID_FRONT_DOOR}&project_id={FRIGATE_NEST_PROJECT_ID}&refresh_token={FRIGATE_NEST_REFRESH_TOKEN}#audio=opus
  webrtc:
    candidates:
      - 10.112.0.4:8555
      - stun:8555

mqtt:
  host: 10.112.0.6
  user: '{FRIGATE_MQTT_USER}'
  password: '{FRIGATE_MQTT_PASSWORD}'

cameras:
  front_yard:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/front_yard
          input_args: preset-rtsp-restream-low-latency
          roles:
            - audio
            - record
            - detect
    detect:
      width: 1280
      height: 960
      fps: 5
    motion:
      mask: 0.017,0.01,0.282,0.012,0.283,0.043,0.016,0.042
      threshold: 40
      contour_area: 30
      improve_contrast: true
    ui:
      order: 1
    zones:
      driveway:
        coordinates: 
          0.605,0.089,0.873,0.146,1,0.42,1,0.998,0.121,0.999,0,0.831,0,0.417,0.08,0.568,0.231,0.369,0.443,0.213
        inertia: 3
      lawn:
        coordinates: 
          0.223,0.369,0,0.337,0,0.206,0.057,0.166,0.123,0.131,0.125,0.15,0.214,0.111,0.324,0.1,0.605,0.088
        inertia: 3
      Front_Porch:
        coordinates: 0.225,0.373,0,0.337,0,0.41,0.08,0.565
        inertia: 3
    snapshots:
      required_zones:
        - driveway
        - lawn
        - Front_Porch
    objects:
      track:
        - person
        - face
        - car
        - license_plate
        - fedex
        - ups
        - amazon
        - package
        - dog
        - cat
    review:
      alerts:
        required_zones:
          - driveway
          - lawn
          - Front_Porch
      detections:
        required_zones:
          - Front_Porch
          - driveway
          - lawn
  back_yard:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/back_yard
          input_args: preset-rtsp-restream-low-latency
          roles:
            - audio
            - record
            - detect
    detect:
      width: 1280
      height: 960
      fps: 5
    motion:
      mask: 0.015,0.011,0.278,0.009,0.278,0.041,0.016,0.042
      threshold: 40
      contour_area: 30
      improve_contrast: true
    ui:
      order: 2
    zones:
      attached_garage_back_door:
        coordinates: 
          1,1,0.797,0.746,0.528,0.44,0.626,0.404,0.633,0.268,0.773,0.354,0.792,0.057,1,0.119,1,0.677
        inertia: 3
      behind_attached_garage:
        coordinates: 0,1,0,0.633,0.529,0.438,0.703,0.641,0.925,0.908,0.87,1
        inertia: 3
      distant_backyard:
        coordinates: 0.631,0.402,0.64,0.117,0.371,0.167,0,0.347,0,0.628
        inertia: 3
      detached_garage_driveway:
        coordinates: 
          0.194,0.112,0,0.252,0.001,0.343,0.37,0.165,0.54,0.133,0.541,0,0.438,0,0.438,0.046
        inertia: 3
    snapshots:
      required_zones:
        - attached_garage_back_door
        - behind_attached_garage
        - distant_backyard
        - detached_garage_driveway
    review:
      alerts:
        required_zones:
          - attached_garage_back_door
          - behind_attached_garage
          - distant_backyard
          - detached_garage_driveway
  rear_garage:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/rear_garage
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
            - detect
    detect:
      width: 1280
      height: 720
      fps: 5
    audio:
      enabled: false
    #motion:
    #  mask:
    #    - 0,0,0,0
    ui:
      order: 3
    #zones:
    #  driveway:
    #   coordinates: 0,0,0,0
    #record:
    #  events:
    #    required_zones:
    #      - driveway
    #      - lawn
    motion:
      threshold: 40
      contour_area: 30
      improve_contrast: true
  cat_room:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cat_room
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
            - detect
    detect:
      width: 1280
      height: 720
      fps: 5
    audio:
      enabled: false
    #motion:
    #  mask:
    #    - 0,0,0,0
    ui:
      order: 4
    #zones:
    #  driveway:
    #   coordinates: 0,0,0,0
    #record:
    #  events:
    #    required_zones:
    #      - driveway
    #      - lawn
    genai:
      enabled: false
    motion:
      threshold: 40
      contour_area: 30
      improve_contrast: true
      mask: 0.425,0.702,0.26,0.675,0.272,0.474,0.437,0.511
  back_yard_sw:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/back_yard_sw
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
            - detect
    detect:
      width: 1280
      height: 720
      fps: 5
    audio:
      enabled: false
    #motion:
    #  mask:
    #    - 0,0,0,0
    ui:
      order: 5
    #zones:
    #  driveway:
    #   coordinates: 0,0,0,0
    #record:
    #  events:
    #    required_zones:
    #      - driveway
    #      - lawn
    motion:
      threshold: 40
      contour_area: 30
      improve_contrast: true
  front_door:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/front_door
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
            - detect
    detect:
      width: 1200
      height: 900
      fps: 5
      annotation_offset: -600
    audio:
      enabled: false
    #motion:
    #  mask:
    #    - 0,0,0,0
    ui:
      order: 6
    #zones:
    #  driveway:
    #   coordinates: 0,0,0,0
    #record:
    #  events:
    #    required_zones:
    #      - driveway
    #      - lawn
    motion:
      threshold: 40
      contour_area: 30
      improve_contrast: true
audio:
  enabled: true
  listen:
    - bark
    - scream
    - speech
    - yell
    - skidding
    - tire_squeal
    - emergency_vehicle
    - smoke_detector
    - fire_alarm
    - gunshot
    - glass
    - crack
    - shatter
    - smash
    - breaking

snapshots:
  enabled: true
  crop: true
  quality: 90
  timestamp: true

record:
  enabled: true
  sync_recordings: false
  retain:
    days: 7
    mode: all
  alerts:
    retain:
      days: 42
  detections:
    retain:
      days: 42
objects:
  track:
    - person
    - face
    - car
    - dog
    - cat
  filters:
    dog:
      min_score: .65
      threshold: .8
    cat:
      min_score: .65
      threshold: .8
    face:
      min_score: .7
    package:
      min_score: .65
      threshold: .9
    license_plate:
      min_score: .6
    amazon:
      min_score: .75
    ups:
      min_score: .75
    fedex:
      min_score: .75
    person:
      min_score: .65
      threshold: .85
    car:
      min_score: .65
      threshold: .85

telemetry:
  network_interfaces:
    - eno
    - br
    - net
  stats:
    network_bandwidth: true

detect:
  stationary:
    interval: 25

motion:
  threshold: 40
  contour_area: 30
  frame_height: 75
  improve_contrast: false
  mqtt_off_delay: 30

genai:
  enabled: true
  provider: gemini
  api_key: '{FRIGATE_GEMINI_API_KEY}'
  model: gemini-1.5-flash

semantic_search:
  enabled: true
  reindex: true

model:
  path: plus://redacted

detectors:
  coral:
    type: edgetpu
    device: usb

ffmpeg:
  output_args:
    record: preset-record-generic-audio-aac
  hwaccel_args: preset-vaapi
camera_groups:
  Front_Yard:
    order: 0
    icon: LuTrees
    cameras:
      - front_yard
      - front_door
  Back_Yard:
    order: 0
    icon: LuTrees
    cameras:
      - rear_garage
      - back_yard_sw
      - back_yard
  Birdseye:
    order: 2
    icon: LuAlertTriangle
    cameras: birdseye
version: 0.15-0
notifications:
  enabled: 'true'
  email: [email protected]

docker-compose file or Docker CLI command

N/A

Relevant Frigate log output

Process embeddings_manager:
2024-10-15 14:50:36.887623-05:002024-10-15 19:50:36.887239175  Traceback (most recent call last):
2024-10-15 14:50:36.887631-05:002024-10-15 19:50:36.887240493    File "/opt/frigate/frigate/util/builtin.py", line 360, in serialize
2024-10-15 14:50:36.887649-05:002024-10-15 19:50:36.887241717      return struct.pack("%sf" % len(vector), *vector)
2024-10-15 14:50:36.887656-05:002024-10-15 19:50:36.887242947  struct.error: required argument is not a float
2024-10-15 14:50:36.887662-05:002024-10-15 19:50:36.887243914  
2024-10-15 14:50:36.887668-05:002024-10-15 19:50:36.887245120  During handling of the above exception, another exception occurred:
2024-10-15 14:50:36.887679-05:002024-10-15 19:50:36.887246026  
2024-10-15 14:50:36.887685-05:002024-10-15 19:50:36.887247029  Traceback (most recent call last):
2024-10-15 14:50:36.887691-05:002024-10-15 19:50:36.887248332    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-10-15 14:50:36.887697-05:002024-10-15 19:50:36.887249340      self.run()
2024-10-15 14:50:36.887703-05:002024-10-15 19:50:36.887250546    File "/opt/frigate/frigate/util/process.py", line 41, in run_wrapper
2024-10-15 14:50:36.887713-05:002024-10-15 19:50:36.887251586      return run(*args, **kwargs)
2024-10-15 14:50:36.887719-05:002024-10-15 19:50:36.887252891    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-10-15 14:50:36.887725-05:002024-10-15 19:50:36.887254016      self._target(*self._args, **self._kwargs)
2024-10-15 14:50:36.887731-05:002024-10-15 19:50:36.887255954    File "/opt/frigate/frigate/embeddings/__init__.py", line 59, in manage_embeddings
2024-10-15 14:50:36.887742-05:002024-10-15 19:50:36.887257008      maintainer = EmbeddingMaintainer(
2024-10-15 14:50:36.887749-05:002024-10-15 19:50:36.887276938    File "/opt/frigate/frigate/embeddings/maintainer.py", line 50, in __init__
2024-10-15 14:50:36.887754-05:002024-10-15 19:50:36.887278110      self.embeddings.reindex()
2024-10-15 14:50:36.887766-05:002024-10-15 19:50:36.887279392    File "/opt/frigate/frigate/embeddings/embeddings.py", line 267, in reindex
2024-10-15 14:50:36.887772-05:002024-10-15 19:50:36.887280588      self.batch_upsert_description(batch_descs)
2024-10-15 14:50:36.887778-05:002024-10-15 19:50:36.887281897    File "/opt/frigate/frigate/embeddings/embeddings.py", line 189, in batch_upsert_description
2024-10-15 14:50:36.887784-05:002024-10-15 19:50:36.887283014      items.append(serialize(embeddings[i]))
2024-10-15 14:50:36.887794-05:002024-10-15 19:50:36.887295061    File "/opt/frigate/frigate/util/builtin.py", line 364, in serialize
2024-10-15 14:50:36.887800-05:002024-10-15 19:50:36.887296315      raise ValueError(f"Failed to pack vector: {e}. Vector: {vector}")
2024-10-15 14:50:36.887806-05:002024-10-15 19:50:36.887297842  ValueError: Failed to pack vector: required argument is not a float. Vector: [array([-2.7319e-01,  1.5796e-01, -1.1945e-01,  7.7515e-02, -6.9031e-02,
2024-10-15 14:50:36.887816-05:002024-10-15 19:50:36.887299116          4.4385e-01,  2.2537e-02, -1.6220e-02,  1.1551e-02,  1.9312e-01,
2024-10-15 14:50:36.887822-05:002024-10-15 19:50:36.887300312          1.6528e-01,  5.1221e-01,  4.2065e-01, -4.4043e-01, -1.7395e-01,
2024-10-15 14:50:36.887829-05:002024-10-15 19:50:36.887301483         -7.7698e-02,  8.5010e-01, -1.0193e-01,  5.7227e-01, -2.6001e-01,
2024-10-15 14:50:36.887839-05:002024-10-15 19:50:36.887314908          6.1084e-01, -2.5055e-02,  8.3801e-02, -1.1346e-01,  3.6475e-01,
2024-10-15 14:50:36.887845-05:002024-10-15 19:50:36.887316122          2.7734e-01,  2.9688e-01,  5.1367e-01, -2.0459e-01,  1.0841e-02,
2024-10-15 14:50:36.887851-05:002024-10-15 19:50:36.887317374         -3.3051e-02, -2.7124e-01, -3.1281e-02, -5.4346e-01, -8.7646e-02,
2024-10-15 14:50:36.887857-05:002024-10-15 19:50:36.887318620         -2.9004e-01,  5.8203e-01, -7.1045e-02, -1.1890e-01, -5.0195e-01,
2024-10-15 14:50:36.887867-05:002024-10-15 19:50:36.887319976          1.7175e-01, -9.0820e-02,  4.2847e-01, -2.4036e-01, -5.2100e-01,
2024-10-15 14:50:36.887873-05:002024-10-15 19:50:36.887321172          3.4326e-01,  8.1665e-02, -1.5002e-01, -5.0476e-02, -2.7026e-01,
2024-10-15 14:50:36.887879-05:002024-10-15 19:50:36.887322310          8.2275e-02, -8.8501e-02, -6.5088e-01, -1.4076e-02,  1.8103e-01,
2024-10-15 14:50:36.887889-05:002024-10-15 19:50:36.887323478          3.9673e-01,  4.5874e-01,  4.5752e-01, -8.2092e-02,  7.5342e-01,
2024-10-15 14:50:36.887895-05:002024-10-15 19:50:36.887324780         -2.7295e-01,  1.0516e-01, -2.9272e-01,  2.1313e-01,  1.1646e-01,
2024-10-15 14:50:36.887901-05:002024-10-15 19:50:36.887325906          3.8428e-01,  1.6931e-01,  1.0144e-01,  6.2744e-02, -1.0669e-01,
2024-10-15 14:50:36.887908-05:002024-10-15 19:50:36.887327029          1.7444e-01, -1.0178e-02, -4.6558e-01,  1.3110e-01, -3.6548e-01,
2024-10-15 14:50:36.887918-05:002024-10-15 19:50:36.887328177          1.1597e-01, -1.5100e-01,  5.7666e-01,  8.8623e-02, -5.3320e-01,
2024-10-15 14:50:36.887924-05:002024-10-15 19:50:36.887329365         -2.7863e-02,  5.9863e-01, -5.1758e-01,  4.1284e-01, -5.7251e-02,
2024-10-15 14:50:36.887930-05:002024-10-15 19:50:36.887344112         -2.4023e-01,  2.7051e-01, -7.3193e-01,  2.9346e-01, -4.2653e-04,
2024-10-15 14:50:36.887942-05:002024-10-15 19:50:36.887345380          5.0476e-02,  5.3076e-01,  2.1094e-01, -6.4453e-02, -7.1045e-02,
2024-10-15 14:50:36.887953-05:002024-10-15 19:50:36.887346535          2.6099e-01, -2.7298e-02, -1.1359e-01, -1.7700e-01,  2.2583e-01,
2024-10-15 14:50:36.887960-05:002024-10-15 19:50:36.887347721         -3.2471e-02,  2.1680e-01, -2.2290e-01, -2.8320e-01,  2.7979e-01,
2024-10-15 14:50:36.887966-05:002024-10-15 19:50:36.887348875         -1.2683e-01, -1.2286e-01,  1.7139e-01,  3.9258e-01,  5.1086e-02,
2024-10-15 14:50:36.887977-05:002024-10-15 19:50:36.887350142         -1.9995e-01, -2.1130e-01, -4.8584e-01, -2.5684e-01, -1.2408e-01,
2024-10-15 14:50:36.887983-05:002024-10-15 19:50:36.887351307         -1.4198e-02, -3.2166e-02, -3.3960e-01, -1.1212e-01, -2.5098e-01,
2024-10-15 14:50:36.887989-05:002024-10-15 19:50:36.887352412          2.9160e-02,  8.5693e-01,  2.8638e-01, -7.7393e-01, -1.6467e-01,
2024-10-15 14:50:36.887999-05:002024-10-15 19:50:36.887353612          9.2529e-02,  1.6394e-01, -7.7209e-02,  7.4463e-01, -7.7454e-02,
2024-10-15 14:50:36.888005-05:002024-10-15 19:50:36.887354806          1.4258e-01, -2.1622e-02,  2.9395e-01,  8.4839e-02, -4.8584e-02,
2024-10-15 14:50:36.888011-05:002024-10-15 19:50:36.887355952          1.6418e-01,  2.0154e-01,  3.6060e-01,  1.9177e-01,  1.6956e-01,
2024-10-15 14:50:36.888017-05:002024-10-15 19:50:36.887386373          2.3889e-01,  1.7883e-01,  2.1350e-01, -5.0488e-01, -1.3831e-01,
2024-10-15 14:50:36.888026-05:002024-10-15 19:50:36.887387679         -1.9714e-01, -7.4036e-02, -1.1792e-01,  3.2788e-01, -2.2583e-01,
2024-10-15 14:50:36.888033-05:002024-10-15 19:50:36.887388871         -9.9121e-02,  5.7526e-02, -4.9316e-01,  2.1558e-01, -3.2300e-01,
2024-10-15 14:50:36.888038-05:002024-10-15 19:50:36.887390046         -6.0205e-01,  1.8616e-01, -4.2305e-03,  3.9795e-01,  4.1431e-01,
2024-10-15 14:50:36.888048-05:002024-10-15 19:50:36.887391177          2.2705e-02, -2.4609e-01, -4.0991e-01,  3.0533e-02, -2.2644e-02,
2024-10-15 14:50:36.888055-05:002024-10-15 19:50:36.887392321         -6.6992e-01, -2.7069e-02, -1.6357e-01,  1.6003e-01, -4.5679e-01,
2024-10-15 14:50:36.888061-05:002024-10-15 19:50:36.887393488         -3.9209e-01,  1.8518e-01,  1.2672e-02, -9.9243e-02,  3.7354e-02,
2024-10-15 14:50:36.888066-05:002024-10-15 19:50:36.887394887          2.8296e-01, -1.2451e+00,  8.5449e-02,  6.0516e-02, -5.3680e-02,
2024-10-15 14:50:36.888076-05:002024-10-15 19:50:36.887396055         -2.4744e-01, -4.5410e-02, -8.7814e-03, -1.8945e-01, -1.5030e-03,
2024-10-15 14:50:36.888082-05:002024-10-15 19:50:36.887397292          3.4082e-01,  5.0977e-01,  3.4619e-01, -4.7722e-03,  1.9421e-01,
2024-10-15 14:50:36.888088-05:002024-10-15 19:50:36.887398453         -4.6216e-01, -9.9548e-02,  1.2415e-01,  2.0154e-01,  8.1787e-02,
2024-10-15 14:50:36.888099-05:002024-10-15 19:50:36.887399652          5.9540e-02, -1.6382e-01,  5.3711e-01,  1.1761e-01,  1.5955e-01,
2024-10-15 14:50:36.888106-05:002024-10-15 19:50:36.887400846         -1.2295e+00,  8.0505e-02,  1.6235e-01, -4.1968e-01,  4.6826e-01,
2024-10-15 14:50:36.888111-05:002024-10-15 19:50:36.887401984         -1.8628e-01, -5.8203e-01,  3.0884e-01, -2.0294e-02, -3.1021e-02,
2024-10-15 14:50:36.888117-05:002024-10-15 19:50:36.887403120          1.2436e-02,  5.4834e-01, -2.0715e-01,  1.5820e-01, -5.5371e-01,
2024-10-15 14:50:36.888127-05:002024-10-15 19:50:36.887404338         -3.3813e-01, -1.7175e-01,  1.3342e-01, -5.6982e-01,  3.6621e-02,
2024-10-15 14:50:36.888134-05:002024-10-15 19:50:36.887405486          3.4814e-01, -1.3367e-01,  5.3613e-01,  3.4521e-01,  1.2952e-01,
2024-10-15 14:50:36.888141-05:002024-10-15 19:50:36.887406675         -2.3230e-01, -2.4072e-01,  1.6394e-01, -4.7638e-02, -2.6196e-01,
2024-10-15 14:50:36.888152-05:002024-10-15 19:50:36.887407847         -2.4548e-03,  5.1172e-01, -4.1473e-02, -6.0608e-02,  1.3782e-01,
2024-10-15 14:50:36.888158-05:002024-10-15 19:50:36.887409001         -4.4586e-02,  1.1682e-01, -4.4739e-02,  3.3008e-01, -2.7686e-01,
2024-10-15 14:50:36.888164-05:002024-10-15 19:50:36.887410142         -3.6719e-01, -4.5020e-01, -3.7817e-01,  9.8877e-02, -9.7418e-04,
2024-10-15 14:50:36.888176-05:002024-10-15 19:50:36.887411354          1.4307e-01, -1.9666e-01,  7.5732e-01, -1.6736e-01,  1.0022e-01,
2024-10-15 14:50:36.888182-05:002024-10-15 19:50:36.887412526         -3.7036e-01, -1.4355e-01,  2.9834e-01, -2.5574e-02,  6.1963e-01,
2024-10-15 14:50:36.888188-05:002024-10-15 19:50:36.887413689         -9.3750e-02, -2.1521e-01, -2.0667e-01, -1.1804e-01, -5.9766e-01,
2024-10-15 14:50:36.888194-05:002024-10-15 19:50:36.887434993         -3.5547e-01, -1.6931e-01,  1.0022e-01, -7.1924e-01, -6.9275e-02,
2024-10-15 14:50:36.888204-05:002024-10-15 19:50:36.887436229         -1.8542e-01, -5.5078e-01,  4.4214e-01,  3.4424e-01,  2.7588e-01,
2024-10-15 14:50:36.888210-05:002024-10-15 19:50:36.887437416         -3.8849e-02, -3.9746e-01,  7.3242e-01,  8.8684e-02, -1.7493e-01,
2024-10-15 14:50:36.888216-05:002024-10-15 19:50:36.887438551         -1.6418e-01,  2.8394e-01, -6.2598e-01, -1.5710e-01, -2.7954e-01,
2024-10-15 14:50:36.888226-05:002024-10-15 19:50:36.887439683         -2.7695e-02, -5.4883e-01,  1.7566e-01, -3.7817e-01,  4.7546e-02,
2024-10-15 14:50:36.888233-05:002024-10-15 19:50:36.887440817          1.1176e-01,  3.1055e-01, -2.6514e-01,  2.3767e-01,  9.8730e-01,
2024-10-15 14:50:36.888249-05:002024-10-15 19:50:36.887441974         -2.1716e-01,  3.7305e-01, -3.3984e-01,  1.1749e-02, -3.3716e-01,
2024-10-15 14:50:36.888267-05:002024-10-15 19:50:36.887443090          2.7710e-01,  1.0750e-02, -3.1372e-01, -4.0479e-01,  2.4707e-01,
2024-10-15 14:50:36.888284-05:002024-10-15 19:50:36.887444222         -2.4182e-01, -2.2754e-01,  4.9048e-01,  9.5276e-02,  2.9053e-01,
2024-10-15 14:50:36.888291-05:002024-10-15 19:50:36.887445361         -2.1729e-01, -1.8359e-01, -1.8909e-01, -7.4097e-02, -1.8848e-01,
2024-10-15 14:50:36.888297-05:002024-10-15 19:50:36.887464623          7.4890e-02, -1.8408e-01,  1.5732e-02, -9.3002e-03,  8.0713e-01,
2024-10-15 14:50:36.888309-05:002024-10-15 19:50:36.887465858          1.3281e-01, -2.4353e-01, -2.1509e-01,  9.1943e-01, -9.2346e-02,
2024-10-15 14:50:36.888316-05:002024-10-15 19:50:36.887467007         -1.1493e-01, -6.6284e-02,  7.1777e-02,  7.9541e-01,  6.5137e-01,
2024-10-15 14:50:36.888322-05:002024-10-15 19:50:36.887468132          2.1460e-01,  4.2529e-01,  1.5930e-01, -2.8882e-01,  5.7922e-02,
2024-10-15 14:50:36.888328-05:002024-10-15 19:50:36.887469313         -4.0015e-01, -5.3253e-02,  6.7529e-01, -3.0859e-01, -9.0515e-02,
2024-10-15 14:50:36.888338-05:002024-10-15 19:50:36.887470478         -1.4636e-01, -4.4434e-01, -2.8662e-01, -5.8174e-03,  2.1924e-01,
2024-10-15 14:50:36.888344-05:002024-10-15 19:50:36.887471627         -1.7242e-02, -4.1931e-02, -8.1604e-02,  4.9780e-01, -2.8052e-01,
2024-10-15 14:50:36.888350-05:002024-10-15 19:50:36.887472856          2.6611e-01, -3.0200e-01,  1.2122e-01, -2.1484e-01,  6.9092e-02,
2024-10-15 14:50:36.888361-05:002024-10-15 19:50:36.887473990          2.8271e-01, -5.9863e-01, -4.7852e-01, -7.1594e-02,  4.2773e-01,
2024-10-15 14:50:36.888367-05:002024-10-15 19:50:36.887475134         -6.7200e-02, -3.2373e-01, -2.0972e-01,  2.5903e-01, -2.9395e-01,
2024-10-15 14:50:36.888373-05:002024-10-15 19:50:36.887476310          6.6101e-02, -6.1035e-01, -3.9111e-01, -7.9407e-02,  4.1357e-01,
2024-10-15 14:50:36.888379-05:002024-10-15 19:50:36.887481233         -1.5114e-02, -4.5410e-01, -4.0479e-01, -2.4561e-01, -4.6899e-01,
2024-10-15 14:50:36.888390-05:002024-10-15 19:50:36.887482487         -7.9269e-03, -3.6792e-01, -2.6733e-01, -7.5073e-02, -2.2681e-01,
2024-10-15 14:50:36.888396-05:002024-10-15 19:50:36.887483693         -1.4978e-01,  4.4922e-01, -2.6929e-01,  2.8491e-01, -3.6102e-02,
2024-10-15 14:50:36.888402-05:002024-10-15 19:50:36.887484835         -3.1372e-01,  4.6753e-01,  8.7036e-02,  9.8572e-02,  3.6890e-01,
2024-10-15 14:50:36.888413-05:002024-10-15 19:50:36.887485977         -1.9824e-01, -2.7197e-01,  1.7136e-02, -2.3303e-01, -3.2275e-01,
2024-10-15 14:50:36.888419-05:002024-10-15 19:50:36.887487158          3.5181e-01, -6.0400e-01, -7.2002e-04,  2.9932e-01, -1.0156e+00,
2024-10-15 14:50:36.888425-05:002024-10-15 19:50:36.887488381         -3.2446e-01,  6.9922e-01,  1.2048e-01, -3.9014e-01, -9.5459e-02,
2024-10-15 14:50:36.888431-05:002024-10-15 19:50:36.887489526          4.3506e-01, -2.6221e-01, -4.1290e-02,  1.2781e-01, -3.2910e-01,
2024-10-15 14:50:36.888442-05:002024-10-15 19:50:36.887490694         -3.2910e-01,  9.0637e-02, -6.9043e-01, -2.7603e-02,  6.0742e-01,
2024-10-15 14:50:36.888448-05:002024-10-15 19:50:36.887491898         -5.1904e-01,  3.8403e-01,  2.3730e-01, -1.0242e-01, -2.0984e-01,
2024-10-15 14:50:36.888454-05:002024-10-15 19:50:36.887493110         -6.3293e-02,  2.6855e-01,  2.9639e-01, -6.9727e-01,  1.7380e-02,
2024-10-15 14:50:36.888464-05:002024-10-15 19:50:36.887494245          1.5735e-01,  1.7480e-01,  3.1909e-01,  5.5762e-01, -1.1530e-01,
2024-10-15 14:50:36.888471-05:002024-10-15 19:50:36.887516248          2.3438e-01,  1.4929e-01, -6.6113e-01, -4.1504e-01,  6.3965e-01,
2024-10-15 14:50:36.888477-05:002024-10-15 19:50:36.887517618         -3.1055e-01,  4.1992e-02,  5.1758e-01,  4.4849e-01, -1.7151e-02,
2024-10-15 14:50:36.888482-05:002024-10-15 19:50:36.887518804          3.5522e-01, -1.1871e-01, -5.0830e-01, -1.3196e-01,  7.1191e-01,
2024-10-15 14:50:36.888493-05:002024-10-15 19:50:36.887519955          2.5803e-02, -2.5513e-01, -1.5747e-01,  2.2107e-01,  4.2529e-01,
2024-10-15 14:50:36.888500-05:002024-10-15 19:50:36.887521123         -4.7144e-01,  1.3855e-01, -3.0640e-01,  1.3618e-02,  3.1616e-01,
2024-10-15 14:50:36.888506-05:002024-10-15 19:50:36.887522283         -6.0400e-01, -1.4478e-01, -2.3511e-01,  1.5930e-01,  6.2012e-01,
2024-10-15 14:50:36.888517-05:002024-10-15 19:50:36.887523503         -5.2734e-01,  3.7964e-01, -7.5500e-02,  7.0068e-02, -1.9189e-01,
2024-10-15 14:50:36.888523-05:002024-10-15 19:50:36.887524618          3.6987e-01, -8.8770e-01, -1.5771e-01, -2.9663e-01, -3.0151e-01,
2024-10-15 14:50:36.888529-05:002024-10-15 19:50:36.887525773          3.4717e-01, -8.2550e-03,  6.0693e-01,  2.3022e-01,  2.8687e-01,
2024-10-15 14:50:36.888544-05:002024-10-15 19:50:36.887545816          2.4976e-01, -2.9248e-01, -1.9202e-01, -6.8994e-01, -5.0830e-01,
2024-10-15 14:50:36.888552-05:002024-10-15 19:50:36.887547202         -2.4951e-01,  1.0046e-01,  2.2473e-01,  1.8896e-01,  1.9397e-01,
2024-10-15 14:50:36.888558-05:002024-10-15 19:50:36.887548430         -6.7444e-02,  1.9336e-01,  4.2578e-01,  2.6660e-01,  6.9287e-01,
2024-10-15 14:50:36.888564-05:002024-10-15 19:50:36.887549622          6.3037e-01, -3.1860e-02, -5.7568e-01, -1.4819e-01,  3.1421e-01,
2024-10-15 14:50:36.888575-05:002024-10-15 19:50:36.887550814         -4.9170e-01, -9.4788e-02,  1.2793e-01, -2.5436e-02, -4.5142e-01,
2024-10-15 14:50:36.888582-05:002024-10-15 19:50:36.887551959          1.4343e-01, -2.0654e-01, -2.1619e-01,  2.0020e-01,  1.3855e-01,
2024-10-15 14:50:36.888588-05:002024-10-15 19:50:36.887553098         -1.6138e-01,  3.2837e-01,  4.6045e-01, -6.5918e-02, -8.3618e-02,
2024-10-15 14:50:36.888600-05:002024-10-15 19:50:36.887554305         -4.1968e-01, -1.6553e-01, -3.6957e-02, -1.9055e-01,  5.0391e-01,
2024-10-15 14:50:36.888606-05:002024-10-15 19:50:36.887555471         -4.9097e-01,  3.9819e-01, -1.2311e-01, -2.3462e-01,  1.0535e-01,
2024-10-15 14:50:36.888612-05:002024-10-15 19:50:36.887556694          3.1641e-01, -5.8319e-02,  5.8632e-03, -1.1725e-01,  2.2949e-01,
2024-10-15 14:50:36.888618-05:002024-10-15 19:50:36.887557862         -5.8008e-01,  1.0199e-01, -7.2363e-01,  3.3228e-01,  2.4866e-01,
2024-10-15 14:50:36.888630-05:002024-10-15 19:50:36.887559092         -2.1741e-01, -2.3755e-01, -9.3155e-03, -7.9407e-02, -2.8589e-01,
2024-10-15 14:50:36.888636-05:002024-10-15 19:50:36.887560288         -1.6464e-02,  1.9482e-01, -1.7737e-01,  2.0361e-01, -3.7109e-01,
2024-10-15 14:50:36.888642-05:002024-10-15 19:50:36.887561523          4.1162e-01,  1.1169e-01, -4.1919e-01, -4.0186e-01,  5.2185e-03,
2024-10-15 14:50:36.888653-05:002024-10-15 19:50:36.887562717          1.8215e-03, -3.6768e-01,  5.1660e-01,  3.0786e-01, -1.8799e-01,
2024-10-15 14:50:36.888659-05:002024-10-15 19:50:36.887563870          8.4045e-02,  6.1096e-02,  8.4961e-02, -2.0117e-01, -1.0535e-01,
2024-10-15 14:50:36.888665-05:002024-10-15 19:50:36.887565045          3.2739e-01,  1.7529e-01, -1.8347e-01, -2.4719e-02, -2.3697e-02,
2024-10-15 14:50:36.888671-05:002024-10-15 19:50:36.887566260          7.1899e-02, -7.1436e-01,  2.8662e-01, -1.6663e-01, -4.3335e-02,
2024-10-15 14:50:36.888682-05:002024-10-15 19:50:36.887567411         -6.4551e-01,  4.2383e-01, -4.0344e-02, -6.8115e-01,  6.2402e-01,
2024-10-15 14:50:36.888688-05:002024-10-15 19:50:36.887568560          1.1169e-01, -1.9250e-01, -2.1826e-01,  3.0078e-01,  5.2832e-01,
2024-10-15 14:50:36.888694-05:002024-10-15 19:50:36.887569779         -8.3984e-02,  1.8347e-01,  5.8203e-01,  5.7568e-01, -3.3276e-01,
2024-10-15 14:50:36.888705-05:002024-10-15 19:50:36.887570937         -1.0559e-01, -3.4937e-01, -6.9043e-01,  7.9834e-02, -2.0935e-01,
2024-10-15 14:50:36.888711-05:002024-10-15 19:50:36.887572085          3.7109e-01, -5.3711e-02,  5.6519e-02,  3.1836e-01, -3.6682e-02,
2024-10-15 14:50:36.888717-05:002024-10-15 19:50:36.887573236          9.7534e-02, -3.4277e-01, -2.5195e-01, -8.6853e-02,  4.9243e-01,
2024-10-15 14:50:36.888723-05:002024-10-15 19:50:36.887574457         -8.4045e-02, -8.4167e-02,  2.8271e-01,  1.2695e-01,  3.6774e-02,
2024-10-15 14:50:36.888734-05:002024-10-15 19:50:36.887575623          5.3760e-01, -1.3232e-01,  4.7461e-01, -6.7676e-01,  7.3779e-01,
2024-10-15 14:50:36.888741-05:002024-10-15 19:50:36.887576815         -1.7136e-02,  8.5742e-01,  3.6914e-01, -3.9111e-01, -6.4880e-02,
2024-10-15 14:50:36.888747-05:002024-10-15 19:50:36.887577955          3.0347e-01, -1.8860e-01, -2.5684e-01, -1.3538e-01,  4.9854e-01,
2024-10-15 14:50:36.888757-05:002024-10-15 19:50:36.887579094          1.6772e-01,  2.1765e-01, -2.3193e-01,  4.4952e-02, -1.5442e-02,
2024-10-15 14:50:36.888764-05:002024-10-15 19:50:36.887580282         -4.2877e-02, -2.1094e-01,  2.6099e-01, -7.3486e-02,  3.0151e-01,
2024-10-15 14:50:36.888770-05:002024-10-15 19:50:36.887581410          2.4316e-01,  8.7509e-03,  2.5659e-01, -1.0236e-01, -5.1346e-03,
2024-10-15 14:50:36.888776-05:002024-10-15 19:50:36.887582571          6.0254e-01,  2.6352e-02, -1.4355e-01,  4.3408e-01, -3.8232e-01,
2024-10-15 14:50:36.888787-05:002024-10-15 19:50:36.887583750         -1.8335e-01, -4.8126e-02,  8.1641e-01, -4.5166e-01, -1.9849e-01,
2024-10-15 14:50:36.888794-05:002024-10-15 19:50:36.887584914         -6.7566e-02,  1.4990e-01,  9.0698e-02,  4.6356e-02,  1.9730e-02,
2024-10-15 14:50:36.888800-05:002024-10-15 19:50:36.887586149         -5.8380e-02, -3.9111e-01,  2.7100e-02,  2.1411e-01, -1.8616e-01,
2024-10-15 14:50:36.888811-05:002024-10-15 19:50:36.887587333          8.5022e-02,  4.3793e-02,  2.8882e-01, -2.0105e-01, -2.4561e-01,
2024-10-15 14:50:36.888817-05:002024-10-15 19:50:36.887588451         -3.2812e-01,  2.3389e-01,  4.9780e-01,  6.0059e-02,  4.7314e-01,
2024-10-15 14:50:36.888823-05:002024-10-15 19:50:36.887589632          3.7256e-01,  9.1113e-01, -1.7053e-01,  7.3926e-01,  2.9565e-01,
2024-10-15 14:50:36.888834-05:002024-10-15 19:50:36.887590759          4.9268e-01,  4.6875e-01, -5.7031e-01, -2.7435e-02, -9.7998e-01,
2024-10-15 14:50:36.888846-05:002024-10-15 19:50:36.887591901         -3.2690e-01, -1.3367e-01, -4.3433e-01, -9.1736e-02, -6.2988e-01,
2024-10-15 14:50:36.888853-05:002024-10-15 19:50:36.887593033          1.0419e-01,  1.2695e-01,  1.8079e-01,  2.0337e-01,  2.5732e-01,
2024-10-15 14:50:36.888859-05:002024-10-15 19:50:36.887594162          2.4512e-01,  2.7539e-01,  1.5039e-01, -4.1675e-01, -4.7241e-01,
2024-10-15 14:50:36.888870-05:002024-10-15 19:50:36.887595304          3.4570e-01, -6.9629e-01,  2.7954e-01, -1.1670e-01, -1.4087e-01,
2024-10-15 14:50:36.888877-05:002024-10-15 19:50:36.887596462          3.4570e-01,  4.5728e-01,  7.0618e-02, -9.6741e-02, -4.2786e-02,
2024-10-15 14:50:36.888883-05:002024-10-15 19:50:36.887597663          3.3252e-01,  5.6732e-02, -1.3418e+00,  6.0498e-01,  3.0371e-01,
2024-10-15 14:50:36.888894-05:002024-10-15 19:50:36.887598857          4.0088e-01, -3.0322e-01,  1.4380e-01,  6.8604e-02,  5.0430e-03,
2024-10-15 14:50:36.888900-05:002024-10-15 19:50:36.887600050          4.2798e-01,  2.7734e-01, -2.1765e-01,  2.2241e-01, -2.5208e-02,
2024-10-15 14:50:36.888906-05:002024-10-15 19:50:36.887601205          3.1738e-01, -7.1777e-02,  1.0809e-01,  5.3076e-01, -1.5793e-02,
2024-10-15 14:50:36.888912-05:002024-10-15 19:50:36.887602351         -3.2654e-02, -5.1074e-01, -1.8967e-02, -4.1064e-01, -5.1123e-01,
2024-10-15 14:50:36.888923-05:002024-10-15 19:50:36.887603547         -1.5356e-01,  1.4819e-01,  3.1372e-01,  1.5186e-01, -4.3945e-01,
2024-10-15 14:50:36.888930-05:002024-10-15 19:50:36.887604730         -4.9609e-01, -1.1993e-01,  6.7725e-01,  1.6589e-01,  4.1553e-01,
2024-10-15 14:50:36.888936-05:002024-10-15 19:50:36.887605897          3.0005e-01,  3.8721e-01, -1.0651e-01,  1.9666e-01,  4.0039e-01,
2024-10-15 14:50:36.888947-05:002024-10-15 19:50:36.887607052         -2.5000e-01,  2.2839e-01, -1.2115e-01,  1.6174e-01,  3.2153e-01,
2024-10-15 14:50:36.888954-05:002024-10-15 19:50:36.887608203          1.9592e-01, -6.0156e-01, -4.1431e-01,  2.3364e-01,  1.5979e-01,
2024-10-15 14:50:36.888960-05:002024-10-15 19:50:36.887609407          1.3965e-01, -6.6211e-01, -1.6235e-01,  3.5034e-01, -1.6418e-01,
2024-10-15 14:50:36.888966-05:002024-10-15 19:50:36.887610612         -1.1383e-02, -4.7046e-01,  1.7090e-01, -6.5723e-01,  5.6030e-02,
2024-10-15 14:50:36.888977-05:002024-10-15 19:50:36.887611732          2.1350e-01, -2.8198e-01, -2.4402e-01,  3.7280e-01, -3.9990e-01,
2024-10-15 14:50:36.888983-05:002024-10-15 19:50:36.887612843         -1.8835e-01, -1.3025e-01, -4.8096e-01,  1.4319e-01, -2.1875e-01,
2024-10-15 14:50:36.888989-05:002024-10-15 19:50:36.887613980          6.7432e-01, -1.0215e+00,  3.3350e-01, -1.9287e-01, -2.2803e-01,
2024-10-15 14:50:36.889000-05:002024-10-15 19:50:36.887615112          2.2278e-01, -3.6133e-01,  2.9614e-01], dtype=float16)]

Relevant go2rtc log output

N/A

Operating system

Debian

Install method

Docker Compose

Network connection

Wired

Camera make and model

N/A

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[NickM-27]

Fixed in #14378

[NickM-27]

in that case we will need specific info about what is actually using memory

[T3chArmy]

Not sure if this is beneficial (also, correction, it's Kubernetes, not Docker). I think in this attempt I bumped resource limit to 8GB.

formatting is a bit off, but maybe this will give some guidance?

Oct 17 21:46:51 tower kernel: frigate.embeddi invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=999
Oct 17 21:46:51 tower kernel: CPU: 0 PID: 2988220 Comm: frigate.embeddi Tainted: P           OE      6.6.32-production+truenas #1
Oct 17 21:46:51 tower kernel: Hardware name: Gigabyte Technology Co., Ltd. Z370 AORUS Gaming 5/Z370 AORUS Gaming 5-CF, BIOS F15 11/13/2021
Oct 17 21:46:51 tower kernel: Call Trace:
Oct 17 21:46:51 tower kernel:  <TASK>
Oct 17 21:46:51 tower kernel:  dump_stack_lvl+0x47/0x60
Oct 17 21:46:51 tower kernel:  dump_header+0x4a/0x1d0
Oct 17 21:46:51 tower kernel:  oom_kill_process+0xf9/0x190
Oct 17 21:46:51 tower kernel:  out_of_memory+0x256/0x540
Oct 17 21:46:51 tower kernel:  mem_cgroup_out_of_memory+0x134/0x150
Oct 17 21:46:51 tower kernel:  try_charge_memcg+0x717/0x800
Oct 17 21:46:51 tower kernel:  charge_memcg+0x36/0xe0
Oct 17 21:46:51 tower kernel:  __mem_cgroup_charge+0x2c/0x80
Oct 17 21:46:51 tower kernel:  do_anonymous_page+0x8d/0x410
Oct 17 21:46:51 tower kernel:  __handle_mm_fault+0xbf1/0xd90
Oct 17 21:46:51 tower kernel:  handle_mm_fault+0x182/0x370
Oct 17 21:46:51 tower kernel:  do_user_addr_fault+0x165/0x660
Oct 17 21:46:51 tower kernel:  exc_page_fault+0x77/0x170
Oct 17 21:46:51 tower kernel:  asm_exc_page_fault+0x26/0x30
Oct 17 21:46:51 tower kernel: RIP: 0033:0x7fe4677deae5
Oct 17 21:46:51 tower kernel: Code: 83 83 00 31 f6 48 85 db 7f 30 e9 46 fe ff ff 66 0f 1f 44 00 00 85 c9 74 4c 8d 8f 00 00 00 38 c1 e2 10 81 e2 00 00 00 80 09 ca <89> 14 b0 48 83 c6 01 48 39 f3 0f 84 1b fe ff ff 0f b7 54 75 00 89
Oct 17 21:46:51 tower kernel: RSP: 002b:00007ffe2a06cc40 EFLAGS: 00010206
Oct 17 21:46:51 tower kernel: RAX: 00007fe12bffb040 RBX: 0000000028a18e8c RCX: 0000000042e80000
Oct 17 21:46:51 tower kernel: RDX: 0000000042e80000 RSI: 0000000002bcb3f0 RDI: 000000000ae80000
Oct 17 21:46:51 tower kernel: RBP: 00007fe28542ee40 R08: 0000000000000000 R09: 0000000000000000
Oct 17 21:46:51 tower kernel: R10: 000000000c8a1788 R11: 0000000000000004 R12: 0000000000000001
Oct 17 21:46:51 tower kernel: R13: 000000000b72b950 R14: 000000000c8a1770 R15: 000000000c8a1788
Oct 17 21:46:51 tower kernel:  </TASK>
Oct 17 21:46:51 tower kernel: memory: usage 8386532kB, limit 8388608kB, failcnt 11755
Oct 17 21:46:51 tower kernel: swap: usage 0kB, limit 9007199254740988kB, failcnt 0
Oct 17 21:46:51 tower kernel: Memory cgroup stats for /kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e359d1_f7ca_4e5f_97be_d218e58e8cbd.slice:
Oct 17 21:46:51 tower kernel: anon 7084576768
Oct 17 21:46:51 tower kernel: file 1469112320
Oct 17 21:46:51 tower kernel: kernel 32759808
Oct 17 21:46:51 tower kernel: kernel_stack 1654784
Oct 17 21:46:51 tower kernel: pagetables 16797696
Oct 17 21:46:51 tower kernel: sec_pagetables 0
Oct 17 21:46:51 tower kernel: percpu 82320
Oct 17 21:46:51 tower kernel: sock 49152
Oct 17 21:46:51 tower kernel: vmalloc 0
Oct 17 21:46:51 tower kernel: shmem 1469091840
Oct 17 21:46:51 tower kernel: zswap 0
Oct 17 21:46:51 tower kernel: zswapped 0
Oct 17 21:46:51 tower kernel: file_mapped 413696
Oct 17 21:46:51 tower kernel: file_dirty 0
Oct 17 21:46:51 tower kernel: file_writeback 20480
Oct 17 21:46:51 tower kernel: swapcached 0
Oct 17 21:46:51 tower kernel: anon_thp 6478102528
Oct 17 21:46:51 tower kernel: file_thp 0
Oct 17 21:46:51 tower kernel: shmem_thp 880803840
Oct 17 21:46:51 tower kernel: inactive_anon 7109271552
Oct 17 21:46:51 tower kernel: active_anon 351731712
Oct 17 21:46:51 tower kernel: inactive_file 12288
Oct 17 21:46:51 tower kernel: active_file 8192
Oct 17 21:46:51 tower kernel: unevictable 1092812800
Oct 17 21:46:51 tower kernel: slab_reclaimable 8849552
Oct 17 21:46:51 tower kernel: slab_unreclaimable 4775496
Oct 17 21:46:51 tower kernel: slab 13625048
Oct 17 21:46:51 tower kernel: workingset_refault_anon 0
Oct 17 21:46:51 tower kernel: workingset_refault_file 10098
Oct 17 21:46:51 tower kernel: workingset_activate_anon 0
Oct 17 21:46:51 tower kernel: workingset_activate_file 0
Oct 17 21:46:51 tower kernel: workingset_restore_anon 0
Oct 17 21:46:51 tower kernel: workingset_restore_file 0
Oct 17 21:46:51 tower kernel: workingset_nodereclaim 0
Oct 17 21:46:51 tower kernel: pgscan 18506
Oct 17 21:46:51 tower kernel: pgsteal 10130
Oct 17 21:46:51 tower kernel: pgscan_kswapd 0
Oct 17 21:46:51 tower kernel: pgscan_direct 18506
Oct 17 21:46:51 tower kernel: pgscan_khugepaged 0
Oct 17 21:46:51 tower kernel: pgsteal_kswapd 0
Oct 17 21:46:51 tower kernel: pgsteal_direct 10130
Oct 17 21:46:51 tower kernel: pgsteal_khugepaged 0
Oct 17 21:46:51 tower kernel: pgfault 16038678
Oct 17 21:46:51 tower kernel: pgmajfault 10163
Oct 17 21:46:51 tower kernel: pgrefill 2401
Oct 17 21:46:51 tower kernel: pgactivate 7880124
Oct 17 21:46:51 tower kernel: pgdeactivate 2401
Oct 17 21:46:51 tower kernel: pglazyfree 0
Oct 17 21:46:51 tower kernel: pglazyfreed 0
Oct 17 21:46:51 tower kernel: zswpin 0
Oct 17 21:46:51 tower kernel: zswpout 0
Oct 17 21:46:51 tower kernel: thp_fault_alloc 7596
Oct 17 21:46:51 tower kernel: thp_collapse_alloc 0
Oct 17 21:46:51 tower kernel: Tasks state (memory values in pages):
Oct 17 21:46:51 tower kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Oct 17 21:46:51 tower kernel: [2947291] 65535 2947291      243       32    28672        0          -998 pause
Oct 17 21:46:51 tower kernel: [2987424]     0 2987424       52        0    28672        0           999 s6-svscan
Oct 17 21:46:51 tower kernel: [2987444]     0 2987444       53        0    24576        0           999 s6-supervise
Oct 17 21:46:51 tower kernel: [2987446]     0 2987446       50        0    24576        0           999 s6-linux-init-s
Oct 17 21:46:51 tower kernel: [2987453]     0 2987453       53        0    28672        0           999 s6-supervise
Oct 17 21:46:51 tower kernel: [2987454]     0 2987454       53        0    24576        0           999 s6-supervise
Oct 17 21:46:51 tower kernel: [2987455]     0 2987455       53        0    28672        0           999 s6-supervise
Oct 17 21:46:51 tower kernel: [2987456]     0 2987456       53        0    28672        0           999 s6-supervise
Oct 17 21:46:51 tower kernel: [2987457]     0 2987457       53        0    28672        0           999 s6-supervise
Oct 17 21:46:51 tower kernel: [2987458]     0 2987458       53        0    28672        0           999 s6-supervise
Oct 17 21:46:51 tower kernel: [2987459]     0 2987459       53        0    28672        0           999 s6-supervise
Oct 17 21:46:51 tower kernel: [2987460]     0 2987460       53        0    28672        0           999 s6-supervise
Oct 17 21:46:51 tower kernel: [2987461]     0 2987461       53        0    28672        0           999 s6-supervise
Oct 17 21:46:51 tower kernel: [2987462]     0 2987462       53        0    28672        0           999 s6-supervise
Oct 17 21:46:51 tower kernel: [2987463]     0 2987463       53        0    28672        0           999 s6-supervise
Oct 17 21:46:51 tower kernel: [2987474]     0 2987474      130        0    28672        0           999 s6-fdholderd
Oct 17 21:46:51 tower kernel: [2987475]     0 2987475       47        0    24576        0           999 s6-ipcserverd
Oct 17 21:46:51 tower kernel: [2987517] 65534 2987517       69        0    36864        0           999 s6-log
Oct 17 21:46:51 tower kernel: [2987518] 65534 2987518       69        0    36864        0           999 s6-log
Oct 17 21:46:51 tower kernel: [2987527]     0 2987527   310633     6488   159744        0           999 go2rtc
Oct 17 21:46:51 tower kernel: [2987536]     0 2987536  1170124    99397  1708032        0           999 python3
Oct 17 21:46:51 tower kernel: [2987892]     0 2987892      622      224    45056        0           999 sleep
Oct 17 21:46:51 tower kernel: [2988215]     0 2988215     3642     2304    65536        0           999 python3
Oct 17 21:46:51 tower kernel: [2988220]     0 2988220  3643623  1678509 14413824        0           999 frigate.embeddi
Oct 17 21:46:51 tower kernel: [2992148]     0 2992148       54        0    24576        0           999 s6-rc
Oct 17 21:46:51 tower kernel: [2992208]     0 2992208       56        0    36864        0           999 s6-svlisten1
Oct 17 21:46:51 tower kernel: [2992210]     0 2992210      114        0    40960        0           999 s6-ftrigrd
Oct 17 21:46:51 tower kernel: oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=cri-containerd-b7d9248ffb054807ed83db1bbbfdb92e179956cc1e6063e92c6853cb3fea98f0.scope,mems_allowed=0,oom_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e359d1_f7ca_4e5f_97be_d218e58e8cbd.slice,task_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e359d1_f7ca_4e5f_97be_d218e58e8cbd.slice/cri-containerd-b7d9248ffb054807ed83db1bbbfdb92e179956cc1e6063e92c6853cb3fea98f0.scope,task=frigate.embeddi,pid=2988220,uid=0

[NickM-27]

how many tracked objects total are you trying to reindex?

[T3chArmy]

Turned resource limits back off for now and restarted the container.

Looks like 10754 tracked objects total.

Another thing I noticed, and maybe it's just coincidence, but when I don't have the Frigate WebUI open, the indexing goes much quicker.

[T3chArmy]

Sorry. I did just notice that I've been running edaccd8 in my testing this evening not the latest build 5d8bcb4.

Just switched to the 5d8bcb4 build just to make sure this wasn't an already resolved issue. Looks like the issue is still occuring in most recent build as well.