This project is a prototype app for analyzing nostr data to help with search, discovery, and onboarding.
[add url] (Note: db contains mostly events from 2023-11-14 to 2023-11-21)
- Vectorizes content with embeddings to enable semantic search across different kinds
- Uses nix to setup infrastructure to:
- Install Nix package manager to setup databases (weaviate and neo4j)
- Start nix:
nix develop
- Download nostr events using strfry
- Export nostr events from strfry database as json (for neo4j import):
strfry export dbdump.jsonl
- Start weaviate docker container:
docker compose up -d
- Create .env file with environment variables:
cp .env.example .env- Set
STRFRY_DB_FOLDER,NEO4J_IMPORT_DIRto correct directories
- Start Neo4j
neo4j start
- Run python script main.py:
python -m conduit.main- create weaviate classes
- load events into weaviate
- load data into neo4j
- Start plotly dash app:
python -m conduit.app
- Go to browser to use app:
-
http://localhost:8050/: home page summarizes the number of events in db
-
Search tab (src): find events related to search phrase using vector-based similarity
-
Network tab (src): show network-level metrics such as most active, followed, and targeted users
-
User tab (src): show user metrics such as histogram of created events (broken down by kind, day of week, and hour of day) and biggest fan (user with the most mentions/reactions to those events)
-
Neo4j tab (src): link to neo4j browser
-
- Strfry: nostr relay
- Weaviate: vector database
- Neo4j: graph database for network analysis
- Plotly Dash: python framework for interactive visualizations
- Rust Nostr SDK python bindings: nostr client library
- TBD