Skip to content

Commit

Permalink
For security. (#3642)
Browse files Browse the repository at this point in the history
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
  • Loading branch information
KevinHuSh authored Nov 26, 2024
1 parent 32cf566 commit 3df1663
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion agent/component/crawler.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
import asyncio
from crawl4ai import AsyncWebCrawler
from agent.component.base import ComponentBase, ComponentParamBase
from api.utils.web_utils import is_valid_url


class CrawlerParam(ComponentParamBase):
Expand All @@ -39,7 +40,7 @@ class Crawler(ComponentBase, ABC):
def _run(self, history, **kwargs):
ans = self.get_input()
ans = " - ".join(ans["content"]) if "content" in ans else ""
if not ans:
if not is_valid_url(ans):
return Crawler.be_output("")
try:
result = asyncio.run(self.get_web(ans))
Expand Down

0 comments on commit 3df1663

Please sign in to comment.