Skip to main content

Use Rust To Reduce The Size Of Your SQLite Database

 Meet sqlite-zstd, a Rust library that compresses your database many fold, leading to great savings in size while conserving its search capabilities intact.


As pointed out in "In Praise Of SQLite", SQLite is not a toy database:


Despite its compact size and absence of the client server model, SQLite is a RDBMS with all the features that make something relational - that is tables, indexes, constraints, triggers, transactions and the like. However, there are few built-in functions compared to PostgreSQL or Oracle. 


SQLite doesn’t have any compression features. This has changed with the Rust-based library, sqlite-zstd which promises to:


provide transparent dictionary-based row-level compression that allows compressing a sqlite database's entries almost as well as if you were compressing the whole DB file, while at the same time retaining random access.


Suffice it to say that we are all aware of the benefits of compressing data, be it that of a PDF document, a humble ZIP file or in this case a database. As a proof of concept and walkthrough of the tool, I'll work with a sample database used by the Joplin note taking app. Well work with Windows because it offers a more straightforward experience.


full article on IProgrammer:

https://www.i-programmer.info/programming/database/15686-use-rust-to-reduce-the-size-of-your-sqlite-database-.html


Comments

Popular posts from this blog

Ingres vs Postgres MVCC Explained With Neo4j's LLM Knowledge Graph Builder

 LLM Knowledge Graph Builder is an application designed to turn unstructured data such as pdfs, text documents, YouTube videos, and web pages, into a knowledge graph stored in Neo4j, promising much better accuracy than simple RAG (Retrieval-Augmented Generation). https://www.i-programmer.info/news/80-java/17967-ingres-vs-postgres-mvcc-explained-with-neo4js-llm-knowledge-graph-builder-.html

The Advent of SQL 2024 Has Commenced

  It's Advent - the time of year when we countdown the days to Christmas - and if your are a programmer complete daily coding challenges with the Advent of Code, the Advent of Perl, the Advent of Java, Javascriptmas, etc. Now we have the Advent of SQL too with 24 SQL challenges to complete before Christmas! https://www.i-programmer.info/news/204-challenges/17678-the-advent-of-sql-2024-has-commenced.html