2 Questions about storage and databases

[thegrillmaster]

Hello everyone, sorry again I have a question. Under high load, Cuckoo and Peekaboo generate large amounts of data, both on the disk and in the DB. Currently I'm using mysql, but it's slow as described by you. So I installed Postgresql and created a DB for Cuckoo. Unfortunately I couldn't do that for Peekaboo. How can I create a DB for Peekaboo?
Then the 2nd question. Is there a script that can delete all databases (i.e. Cuckoo Storage-Directorys and databases) for example older than 3 days back? Unfortunately scripts and DBs are not my strong points... :( Thanks again for your support and the ingenious project PeekabooAV!

[Jack28]

Hi @thegrillmaster,

ragarding your first question:
For a quick test it can be as easy as:
docker run --name some-mariadb -P -e MYSQL_ROOT_PASSWORD=my-secret-pw -e MYSQL_DATABASE=peekaboo -d mariadb
and point peekaboo.conf:url at
url : mysql+mysqldb://root:my-secret-pw@some-mariadb/peekaboo

Or for your PostrgreSQL setup:
in psql

CREATE USER peekaboo WITH PASSWORD 'cuw2K_ah4bu1Kiequ';
CREATE DATABASE peekaboo;
GRANT ALL PRIVILEGES ON DATABASE peekaboo to peekaboo;
\q

and test with $ psql -d peekaboo -U peekaboo
Then change peekaboo.conf to
url : postgresql://peekaboo:cuw2K_ah4bu1Kiequ@127.0.0.1:5432/peekaboo

Peekaboo will then automatically create all tables.

Regarding your second question:
I plan to include this feature in peekaboo-util (#107).
Until then you can use a "DELETE FROM ..." query on your database. Could even be a cron job in the shape of:
0 0 * * * psql -d peekaboo -U peekaboo -c "DELETE FROM analysis_jobs_v6 WHERE analyses_time < NOW() - INTERVAL 14 DAY; "
I'm not sure if this will work. @Clevero mentioned in #90 that he uses SQL statements to achieve this. Maybe he can share them with you.

[thegrillmaster]

Thank you very much for your answer! That will get me further!

[Clevero]

Sorry for the late reply, regarding cleaning the generated data on disk by cuckoo, you can run cuckoo clean

su peekaboo -s /bin/bash
/opt/cuckoo/bin/cuckoo clean

exit

# restart cuckoo services to catch up the cleaned database:
service cuckoo-sandbox restart
service cuckoohttpd restart
service cuckooapi restart

The docs of cuckoo states:
https://cuckoo.readthedocs.io/en/latest/usage/clean/

  To sum up, this command does the following:

    Delete analysis results.
    Delete submitted binaries.
    Delete all associated information of the tasks and samples in the configured database.
    Delete all data in the configured MongoDB database (if configured and enabled in $CWD/conf/reporting.conf).
    Delete all data in the configured ElasticSearch database (if configured and enabled in $CWD/conf/reporting.conf).

Warning

If you use this command you will permanently delete all data stored by Cuckoo in all available storages: the file system, the SQL database, the MongoDB database, and the ElasticSearch database. Use it only if you are sure you would clean up all the data.

To run this procedure (cleaning all cuckoo related stuff) every 3 days, you could create a cronjob that calls the cuckoo clean command every 3 days and restarts the cuckoo services after that

[thegrillmaster]

great! many thanks!