0.16 Hybrid - Object speed synced

docs/docs/configuration/live.md

@@ -3,7 +3,7 @@ id: live
 title: Live View
 ---
 
-Frigate intelligently displays your camera streams on the Live view dashboard. Your camera images update once per minute when no detectable activity is occurring to conserve bandwidth and resources. As soon as any motion is detected, cameras seamlessly switch to a live stream.
+Frigate intelligently displays your camera streams on the Live view dashboard. By default, Frigate employs "smart streaming" where camera images update once per minute when no detectable activity is occurring to conserve bandwidth and resources. As soon as any motion or objects are detected, cameras seamlessly switch to a live stream.
 
 ## Live View technologies
 
@@ -51,9 +51,15 @@ go2rtc:
       - ffmpeg:rtsp://192.168.1.5:554/live0#video=copy
 ```
 
-### Setting Stream For Live UI
+### Setting Streams For Live UI
 
-There may be some cameras that you would prefer to use the sub stream for live view, but the main stream for recording. This can be done via `live -> stream_name`.
+In Frigate 0.16 and later, you can edit your configuration to allow manual selection of the stream you want to view in the Live UI. For example, you may want to view your camera's substream on mobile devices, but the full resolution stream on desktop devices. Setting the `live -> streams` list will populate a dropdown in the UI's Live view that allows you to choose between the streams. This settings is _per device_ and is saved in your device's local storage.
+
+Additionally, when creating and editing camera groups in the UI, you can choose the stream you want to use for your camera group's Live dashboard. The default dashboard ("All Cameras") will always use the first entry you've defined in `streams:` for streaming.
+
+Configure the `streams` option with a "friendly name" for your stream followed by the go2rtc stream name.
+
+Go2rtc is required to use this feature. You cannot specify paths in the `streams` list, only go2rtc stream names.
 
 ```yaml
 go2rtc:
@@ -80,7 +86,9 @@ cameras:
           roles:
             - detect
     live:
-      stream_name: test_cam_sub
+      streams: # <--- Multiple streams for Frigate 0.16 and later
+        - Main Stream: test_cam
+        - Sub Stream: test_cam_sub
 ```
 
 ### WebRTC extra configuration:
@@ -138,3 +146,20 @@ services:
 :::
 
 See [go2rtc WebRTC docs](https://github.com/AlexxIT/go2rtc/tree/v1.8.3#module-webrtc) for more information about this.
+
+### Streaming options on camera group dashboards
+
+Frigate 0.16 and later provides a dialog in the Camera Group Edit pane with several options for streaming on a camera group's dashboard. These settings are _per device_ and are saved in your device's local storage.
+
+- Stream selection using the `live -> streams` configuration option (see _Setting Streams For Live UI_ above)
+- Streaming type:
+  - _No streaming_: Camera images will only update once per minute and no live streaming will occur.
+  - _Smart Streaming_ (default, recommended setting): Smart streaming will update your camera image once per minute when no detectable activity is occurring to conserve bandwidth and resources, since a static picture is the same as a streaming image with no motion or objects. When motion or objects are detected, the image seamlessly switches to a live stream.
+  - _Continuous Streaming_: Camera image will always be a live stream when visible on the dashboard, even if no activity is being detected. Continuous streaming may cause high bandwidth usage and performance issues. **Use with caution.**
+- _Compatibility mode_: Enable this option only if your camera's live stream is displaying color artifacts and has a diagonal line on the right side of the image. Before enabling this, try setting your camera's `detect` width and height to a standard aspect ratio (for example: 640x352 becomes 640x360, and 800x443 becomes 800x450, 2688x1520 becomes 2688x1512, etc). Depending on your browser and device, more than a few cameras in compatibility mode may not be supported, so only use this option if changing your config fails to resolve the color artifacts and diagonal line.
+
+:::note
+
+The default dashboard ("All Cameras") will always use Smart Streaming and the first entry set in your `streams` configuration, if defined. Use a camera group if you need to change any of these settings from these defaults.
+
+:::

docs/docs/configuration/reference.md

@@ -551,10 +551,11 @@ go2rtc:
 # Optional: Live stream configuration for WebUI.
 # NOTE: Can be overridden at the camera level
 live:
-  # Optional: Set the name of the stream configured in go2rtc
+  # Optional: Set the streams configured in go2rtc
   # that should be used for live view in frigate WebUI. (default: name of camera)
   # NOTE: In most cases this should be set at the camera level only.
-  stream_name: camera_name
+  streams:
+    - friendly_name: stream_name
   # Optional: Set the height of the jsmpeg stream. (default: 720)
   # This must be less than or equal to the height of the detect stream. Lower resolutions
   # reduce bandwidth required for viewing the jsmpeg stream. Width is computed to match known aspect ratio.
@@ -639,7 +640,10 @@ cameras:
       front_steps:
         # Required: List of x,y coordinates to define the polygon of the zone.
         # NOTE: Presence in a zone is evaluated only based on the bottom center of the objects bounding box.
-        coordinates: 0.284,0.997,0.389,0.869,0.410,0.745
+        coordinates: 0.033,0.306,0.324,0.138,0.439,0.185,0.042,0.428
+        # Optional: The real-world distances of a 4-sided zone used for zones with speed estimation enabled (default: none)
+        # List distances in order of the zone points coordinates and use the unit system defined in the ui config
+        distances: 10,15,12,11
         # Optional: Number of consecutive frames required for object to be considered present in the zone (default: shown below).
         inertia: 3
         # Optional: Number of seconds that an object must loiter to be considered in the zone (default: shown below)
@@ -788,6 +792,9 @@ ui:
   # https://www.gnu.org/software/libc/manual/html_node/Formatting-Calendar-Time.html
   # possible values are shown above (default: not set)
   strftime_fmt: "%Y/%m/%d %H:%M"
+  # Optional: Set the unit system to either "imperial" or "metric" (default: metric)
+  # Used in the UI and in MQTT topics
+  unit_system: metric
 
 # Optional: Telemetry configuration
 telemetry:

docs/docs/configuration/zones.md

@@ -122,16 +122,42 @@ cameras:
           - car
 ```
 
-### Loitering Time
+### Speed Estimation
 
-Zones support a `loitering_time` configuration which can be used to only consider an object as part of a zone if they loiter in the zone for the specified number of seconds. This can be used, for example, to create alerts for cars that stop on the street but not cars that just drive past your camera.
+Frigate can be configured to estimate the speed of objects moving through a zone. This works by combining data from Frigate's object tracker and "real world" distance measurements of the edges of the zone. The recommended use case for this feature is to track the speed of vehicles on a road.
+
+Your zone must be defined with exactly 4 points and should be aligned to the ground where objects are moving.
+
+![Ground plane 4-point zone](/img/ground-plane.jpg)
+
+Speed estimation requires a minimum number of frames for your object to be tracked before a valid estimate can be calculated, so create your zone away from the edges of the frame for the best results. _Your zone should not take up the full frame._ Once an object enters a speed estimation zone, its speed will continue to be tracked, even after it leaves the zone.
+
+Accurate real-world distance measurements are required to estimate speeds. These distances can be specified in your zone config through the `distances` field.
 
 ```yaml
 cameras:
   name_of_your_camera:
     zones:
-      front_yard:
-        loitering_time: 5 # unit is in seconds
-        objects:
-          - person
+      street:
+        coordinates: 0.033,0.306,0.324,0.138,0.439,0.185,0.042,0.428
+        distances: 10,12,11,13.5
 ```
+
+Each number in the `distance` field represents the real-world distance between the points in the `coordinates` list. So in the example above, the distance between the first two points ([0.033,0.306] and [0.324,0.138]) is 10. The distance between the second and third set of points ([0.324,0.138] and [0.439,0.185]) is 12, and so on. The fastest and most accurate way to configure this is through the Zone Editor in the Frigate UI.
+
+The `distance` values are measured in meters or feet, depending on how `unit_system` is configured in your `ui` config:
+
+```yaml
+ui:
+  # can be "metric" or "imperial", default is metric
+  unit_system: metric
+```
+
+The average and maximum speed during the object's lifetime is saved in Frigate's database and can be seen in the UI in the Tracked Object Details pane in Explore. Current estimated speed can also be seen on the debug view as the third value in the object label. Current estimated speed, average estimated speed, max estimated speed, and velocity angle (the angle of the direction the object is moving relative to the frame) of tracked objects is also sent through the `events` MQTT topic. See the [MQTT docs](../integrations/mqtt.md#frigateevents). These speed values are output as a number in miles per hour (mph) or kilometers per hour (kph), depending on how `unit_system` is configured in your `ui` config.
+
+#### Best practices and caveats
+
+- Speed estimation works best with a straight road or path when your object travels in a straight line across that path. If your object makes turns, speed estimation may not be accurate.
+- Create a zone where the bottom center of your object's bounding box travels directly through it.
+- The more accurate your real-world dimensions can be measured, the more accurate speed estimation will be. However, due to the way Frigate's tracking algorithm works, you may need to tweak the real-world distance values so that estimated speeds better match real-world speeds.
+- The speeds are only an _estimation_ and are highly dependent on camera position, zone points, and real-world measurements. This feature should not be used for law enforcement.

docs/docs/integrations/mqtt.md

@@ -52,7 +52,10 @@ Message published for each changed tracked object. The first message is publishe
     "attributes": {
       "face": 0.64
     }, // attributes with top score that have been identified on the object at any point
-    "current_attributes": [] // detailed data about the current attributes in this frame
+    "current_attributes": [], // detailed data about the current attributes in this frame
+    "estimated_speed": 0.71, // current estimated speed (mph or kph) for objects moving through zones with speed estimation enabled
+    "max_estimated_speed": 1.2, // max estimated speed (mph or kph) for objects moving through zones with speed estimation enabled
+    "velocity_angle": 180 // direction of travel relative to the frame for objects moving through zones with speed estimation enabled
   },
   "after": {
     "id": "1607123955.475377-mxklsc",
@@ -89,7 +92,10 @@ Message published for each changed tracked object. The first message is publishe
         "box": [442, 506, 534, 524],
         "score": 0.86
       }
-    ]
+    ],
+    "estimated_speed": 0.77, // current estimated speed (mph or kph) for objects moving through zones with speed estimation enabled
+    "max_estimated_speed": 1.2, // max estimated speed (mph or kph) for objects moving through zones with speed estimation enabled
+    "velocity_angle": 180 // direction of travel relative to the frame for objects moving through zones with speed estimation enabled
   }
 }
 ```

frigate/api/event.py

@@ -316,7 +316,15 @@ def process_events():
                     k: v
                     for k, v in event.data.items()
                     if k
-                    in ["type", "score", "top_score", "description", "sub_label_score"]
+                    in [
+                        "type",
+                        "score",
+                        "top_score",
+                        "description",
+                        "sub_label_score",
+                        "average_estimated_speed",
+                        "max_estimated_speed",
+                    ]
                 },
                 "event_count": label_counts[event.label],
             }
@@ -581,7 +589,16 @@ def events_search(request: Request, params: EventsSearchQueryParams = Depends())
         processed_event["data"] = {
             k: v
             for k, v in event["data"].items()
-            if k in ["type", "score", "top_score", "description"]
+            if k
+            in [
+                "type",
+                "score",
+                "top_score",
+                "description",
+                "sub_label_score",
+                "average_estimated_speed",
+                "max_estimated_speed",
+            ]
         }
 
         if event["id"] in search_results:

frigate/config/camera/live.py

@@ -1,3 +1,5 @@
+from typing import Dict
+
 from pydantic import Field
 
 from ..base import FrigateBaseModel
@@ -6,6 +8,9 @@
 
 
 class CameraLiveConfig(FrigateBaseModel):
-    stream_name: str = Field(default="", title="Name of restream to use as live view.")
+    streams: Dict[str, str] = Field(
+        default_factory=list,
+        title="Friendly names and restream names to use for live view.",
+    )
     height: int = Field(default=720, title="Live camera view height")
     quality: int = Field(default=8, ge=1, le=31, title="Live camera view quality")

frigate/config/camera/zone.py

@@ -16,6 +16,10 @@ class ZoneConfig(BaseModel):
     coordinates: Union[str, list[str]] = Field(
         title="Coordinates polygon for the defined zone."
     )
+    distances: Optional[Union[str, list[str]]] = Field(
+        default_factory=list,
+        title="Real-world distances for the sides of quadrilateral for the defined zone.",
+    )
     inertia: int = Field(
         default=3,
         title="Number of consecutive frames required for object to be considered present in the zone.",
@@ -49,6 +53,24 @@ def validate_objects(cls, v):
 
         return v
 
+    @field_validator("distances", mode="before")
+    @classmethod
+    def validate_distances(cls, v):
+        if v is None:
+            return None
+
+        if isinstance(v, str):
+            distances = list(map(str, map(float, v.split(","))))
+        elif isinstance(v, list):
+            distances = [str(float(val)) for val in v]
+        else:
+            raise ValueError("Invalid type for distances")
+
+        if len(distances) != 4:
+            raise ValueError("distances must have exactly 4 values")
+
+        return distances
+
     def __init__(self, **config):
         super().__init__(**config)
 

frigate/config/config.py

@@ -176,17 +176,18 @@ def verify_config_roles(camera_config: CameraConfig) -> None:
         )
 
 
-def verify_valid_live_stream_name(
+def verify_valid_live_stream_names(
     frigate_config: FrigateConfig, camera_config: CameraConfig
 ) -> ValueError | None:
     """Verify that a restream exists to use for live view."""
-    if (
-        camera_config.live.stream_name
-        not in frigate_config.go2rtc.model_dump().get("streams", {}).keys()
-    ):
-        return ValueError(
-            f"No restream with name {camera_config.live.stream_name} exists for camera {camera_config.name}."
-        )
+    for _, stream_name in camera_config.live.streams.items():
+        if (
+            stream_name
+            not in frigate_config.go2rtc.model_dump().get("streams", {}).keys()
+        ):
+            return ValueError(
+                f"No restream with name {stream_name} exists for camera {camera_config.name}."
+            )
 
 
 def verify_recording_retention(camera_config: CameraConfig) -> None:
@@ -562,15 +563,15 @@ def post_validation(self, info: ValidationInfo) -> Self:
                     zone.generate_contour(camera_config.frame_shape)
 
             # Set live view stream if none is set
-            if not camera_config.live.stream_name:
-                camera_config.live.stream_name = name
+            if not camera_config.live.streams:
+                camera_config.live.streams = {name: name}
 
             # generate the ffmpeg commands
             camera_config.create_ffmpeg_cmds()
             self.cameras[name] = camera_config
 
             verify_config_roles(camera_config)
-            verify_valid_live_stream_name(self, camera_config)
+            verify_valid_live_stream_names(self, camera_config)
             verify_recording_retention(camera_config)
             verify_recording_segments_setup_with_reasonable_time(camera_config)
             verify_zone_objects_are_tracked(camera_config)

frigate/config/ui.py

@@ -5,7 +5,7 @@
 
 from .base import FrigateBaseModel
 
-__all__ = ["TimeFormatEnum", "DateTimeStyleEnum", "UIConfig"]
+__all__ = ["TimeFormatEnum", "DateTimeStyleEnum", "UnitSystemEnum", "UIConfig"]
 
 
 class TimeFormatEnum(str, Enum):
@@ -21,6 +21,11 @@ class DateTimeStyleEnum(str, Enum):
     short = "short"
 
 
+class UnitSystemEnum(str, Enum):
+    imperial = "imperial"
+    metric = "metric"
+
+
 class UIConfig(FrigateBaseModel):
     timezone: Optional[str] = Field(default=None, title="Override UI timezone.")
     time_format: TimeFormatEnum = Field(
@@ -35,3 +40,6 @@ class UIConfig(FrigateBaseModel):
     strftime_fmt: Optional[str] = Field(
         default=None, title="Override date and time format using strftime syntax."
     )
+    unit_system: UnitSystemEnum = Field(
+        default=UnitSystemEnum.metric, title="The unit system to use for measurements."
+    )

frigate/events/maintainer.py

@@ -25,6 +25,9 @@ def should_update_db(prev_event: Event, current_event: Event) -> bool:
             or prev_event["entered_zones"] != current_event["entered_zones"]
             or prev_event["thumbnail"] != current_event["thumbnail"]
             or prev_event["end_time"] != current_event["end_time"]
+            or prev_event["average_estimated_speed"]
+            != current_event["average_estimated_speed"]
+            or prev_event["max_estimated_speed"] != current_event["max_estimated_speed"]
         ):
             return True
     return False
@@ -210,6 +213,8 @@ def handle_object_detection(
                     "score": score,
                     "top_score": event_data["top_score"],
                     "attributes": attributes,
+                    "average_estimated_speed": event_data["average_estimated_speed"],
+                    "max_estimated_speed": event_data["max_estimated_speed"],
                     "type": "object",
                     "max_severity": event_data.get("max_severity"),
                 },

frigate/object_processing.py

@@ -162,7 +162,12 @@ def get_current_frame(self, draw_options={}):
                     box[2],
                     box[3],
                     text,
-                    f"{obj['score']:.0%} {int(obj['area'])}",
+                    f"{obj['score']:.0%} {int(obj['area'])}"
+                    + (
+                        f" {float(obj['estimated_speed']):.1f}"
+                        if obj["estimated_speed"] != 0
+                        else ""
+                    ),
                     thickness=thickness,
                     color=color,
                 )
@@ -256,6 +261,7 @@ def update(
             new_obj = tracked_objects[id] = TrackedObject(
                 self.config.model,
                 self.camera_config,
+                self.config.ui,
                 self.frame_cache,
                 current_detections[id],
             )