Skip to content

database.md

Latest commit

 

History

History
188 lines (138 loc) · 4.69 KB

database.md

File metadata and controls

188 lines (138 loc) · 4.69 KB

Neo4j Specific Database Queries

Check Neo4j version

$ VER=$(docker exec moltbook-neo4j neo4j --version | awk '{print $NF}')
$ echo $VER
5.26.16

Check Database Size

$ docker exec moltbook-neo4j sh -c 'du -sh /data /data/databases /data/transactions 2>/dev/null'
1.1G    /data

$ docker exec moltbook-neo4j sh -c 'du -sh /data/databases/neo4j /data/databases/system /data/transactions/neo4j /data/transactions/system 2>/dev/null'
312M    /data/databases/neo4j
1.2M    /data/databases/system
514M    /data/transactions/neo4j
258M    /data/transactions/system

Download Database dump

# 1) Get the exact Neo4j version running in your container
VER=$(docker exec moltbook-neo4j neo4j --version | awk '{print $NF}')

# 2) Make a local folder for dump files
mkdir -p "$HOME/neo4j_backups"

# 3) Stop the running Neo4j container (required for Community/offline dump)
docker stop moltbook-neo4j

# 4) Dump the main database
docker run --rm -it \
  -v moltbook_neo4j_data:/data \
  -v "$HOME/neo4j_backups":/backups \
  neo4j/neo4j-admin:$VER \
  neo4j-admin database dump neo4j --to-path=/backups --overwrite-destination=true

# 5) Dump the system database too
docker run --rm -it \
  -v moltbook_neo4j_data:/data \
  -v "$HOME/neo4j_backups":/backups \
  neo4j/neo4j-admin:$VER \
  neo4j-admin database dump system --to-path=/backups --overwrite-destination=true

# 6) Start Neo4j again
docker start moltbook-neo4j
  • If doesn't work due to permission issue:
# 1) Get the exact Neo4j version running in your container
VER=$(docker exec moltbook-neo4j neo4j --version | awk '{print $NF}')

# 2) Make a local folder for dump files
mkdir -p "$HOME/neo4j_backups"

# 3) Stop the running Neo4j container (required for Community/offline dump)
docker stop moltbook-neo4j

# 4) create /data/neo4j.dump inside the Neo4j data volume
docker run --rm -it \
  --user=0:0 \
  -v moltbook_neo4j_data:/data \
  neo4j:${VER}-community \
  sh -c 'neo4j-admin database dump neo4j --to-path=/data --overwrite-destination=true && ls -lh /data/*.dump'

# 5) Copy it out to your current directory
docker create --name neo4j-dump-copy -v moltbook_neo4j_data:/data alpine true
docker cp neo4j-dump-copy:/data/neo4j.dump ./neo4j.dump
docker rm neo4j-dump-copy
ls -lh ./neo4j.dump

# 6) Copy out the system database
docker run --rm -it \
  --user=0:0 \
  -v moltbook_neo4j_data:/data \
  neo4j:${VER}-community \
  sh -c 'neo4j-admin database dump system --to-path=/data --overwrite-destination=true && ls -lh /data/system.dump'

docker create --name neo4j-dump-copy -v moltbook_neo4j_data:/data alpine true
docker cp neo4j-dump-copy:/data/system.dump ./system.dump
docker rm neo4j-dump-copy

docker start moltbook-neo4j

  • Backup location: $HOME/neo4j_backups/neo4j.dump and $HOME/neo4j_backups/system.dump

  • To restore

docker stop moltbook-neo4j

docker run --rm -it \
  --user=0:0 \
  -v moltbook_neo4j_data:/data \
  -v "$PWD:/backups:Z" \
  neo4j:${VER}-community \
  sh -c '
    neo4j-admin database load system --from-path=/backups --overwrite-destination=true &&
    neo4j-admin database load neo4j --from-path=/backups --overwrite-destination=true
  '

docker start moltbook-neo4j

Apply Schema Updates (without wiping volumes)

To add new indexes/constraints, just re-run schema. Because statements use IF NOT EXISTS, this is safe.

Option A (recommended): rerun init script

docker compose run --rm crawler python -m scripts.init_db

Option B: apply schema directly with cypher-shell inside the Neo4j container

# Find the Neo4j container name
docker ps --format '{{.Names}}' | grep neo4j

# Then apply schema
docker exec -i moltbook-neo4j cypher-shell -u neo4j -p "$NEO4J_PASSWORD" < crawler/cypher/schema.cypher

Clear Database and Start Over

Delete all nodes/edges (keeps schema)

Run in Neo4j Browser:

MATCH (n) DETACH DELETE n;

Then re-run full crawl.

Common Queries / Update Checks

High-level counts

MATCH (n) RETURN labels(n) AS label, count(*) AS cnt ORDER BY cnt DESC;
MATCH ()-[r]->() RETURN type(r) AS rel, count(*) AS cnt ORDER BY cnt DESC;

Top submolts by number of posts observed

MATCH (s:Submolt)<-[:IN_SUBMOLT]-(p:Post)
RETURN s.name, count(p) AS posts
ORDER BY posts DESC
LIMIT 20;

Recently active agents

MATCH (a:Agent)
WHERE a.last_active IS NOT NULL
RETURN a.name, a.karma, a.last_active
ORDER BY a.last_active DESC
LIMIT 25;

Agents needing profile refresh

MATCH (a:Agent)
WHERE a.profile_last_fetched_at IS NULL
   OR a.profile_last_fetched_at < datetime() - duration("P7D")
RETURN a.name, a.profile_last_fetched_at
ORDER BY a.profile_last_fetched_at ASC
LIMIT 200;