Skip to content

Commit

Permalink
Genai descriptions are not generated until tracked objects end (blake…
Browse files Browse the repository at this point in the history
  • Loading branch information
hawkeye217 authored Dec 17, 2024
1 parent d9ef8fa commit 3dc26e7
Show file tree
Hide file tree
Showing 2 changed files with 81 additions and 49 deletions.
2 changes: 2 additions & 0 deletions docs/docs/configuration/genai.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ title: Generative AI

Generative AI can be used to automatically generate descriptive text based on the thumbnails of your tracked objects. This helps with [Semantic Search](/configuration/semantic_search) in Frigate to provide more context about your tracked objects. Descriptions are accessed via the _Explore_ view in the Frigate UI by clicking on a tracked object's thumbnail.

Requests for a description are sent off automatically to your AI provider at the end of the tracked object's lifecycle. Descriptions can also be regenerated manually via the Frigate UI.

:::info

Semantic Search must be enabled to use Generative AI.
Expand Down
128 changes: 79 additions & 49 deletions web/src/components/overlay/detail/SearchDetailDialog.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -469,60 +469,90 @@ function ObjectDetailsTab({
</div>
</div>
<div className="flex flex-col gap-1.5">
<div className="text-sm text-primary/40">Description</div>
<Textarea
className="h-64"
placeholder="Description of the tracked object"
value={desc}
onChange={(e) => setDesc(e.target.value)}
/>
{config?.cameras[search.camera].genai.enabled &&
!search.end_time &&
(config.cameras[search.camera].genai.required_zones.length === 0 ||
search.zones.some((zone) =>
config.cameras[search.camera].genai.required_zones.includes(zone),
)) &&
(config.cameras[search.camera].genai.objects.length === 0 ||
config.cameras[search.camera].genai.objects.includes(
search.label,
)) ? (
<>
<div className="text-sm text-primary/40">Description</div>
<div className="flex h-64 flex-col items-center justify-center gap-3 border p-4 text-sm text-primary/40">
<div className="flex">
<ActivityIndicator />
</div>
<div className="flex">
Frigate will not request a description from your Generative AI
provider until the tracked object's lifecycle has ended.
</div>
</div>
</>
) : (
<>
<div className="text-sm text-primary/40">Description</div>
<Textarea
className="h-64"
placeholder="Description of the tracked object"
value={desc}
onChange={(e) => setDesc(e.target.value)}
/>
</>
)}

<div className="flex w-full flex-row justify-end gap-2">
{config?.cameras[search.camera].genai.enabled && (
<div className="flex items-center">
{config?.cameras[search.camera].genai.enabled && search.end_time && (
<>
<div className="flex items-start">
<Button
className="rounded-r-none border-r-0"
aria-label="Regenerate tracked object description"
onClick={() => regenerateDescription("thumbnails")}
>
Regenerate
</Button>
{search.has_snapshot && (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
className="rounded-l-none border-l-0 px-2"
aria-label="Expand regeneration menu"
>
<FaChevronDown className="size-3" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuItem
className="cursor-pointer"
aria-label="Regenerate from snapshot"
onClick={() => regenerateDescription("snapshot")}
>
Regenerate from Snapshot
</DropdownMenuItem>
<DropdownMenuItem
className="cursor-pointer"
aria-label="Regenerate from thumbnails"
onClick={() => regenerateDescription("thumbnails")}
>
Regenerate from Thumbnails
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
)}
</div>

<Button
className="rounded-r-none border-r-0"
aria-label="Regenerate tracked object description"
onClick={() => regenerateDescription("thumbnails")}
variant="select"
aria-label="Save"
onClick={updateDescription}
>
Regenerate
Save
</Button>
{search.has_snapshot && (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
className="rounded-l-none border-l-0 px-2"
aria-label="Expand regeneration menu"
>
<FaChevronDown className="size-3" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuItem
className="cursor-pointer"
aria-label="Regenerate from snapshot"
onClick={() => regenerateDescription("snapshot")}
>
Regenerate from Snapshot
</DropdownMenuItem>
<DropdownMenuItem
className="cursor-pointer"
aria-label="Regenerate from thumbnails"
onClick={() => regenerateDescription("thumbnails")}
>
Regenerate from Thumbnails
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
)}
</div>
</>
)}
<Button
variant="select"
aria-label="Save"
onClick={updateDescription}
>
Save
</Button>
</div>
</div>
</div>
Expand Down

0 comments on commit 3dc26e7

Please sign in to comment.