Skip to main content

Use Rust To Reduce The Size Of Your SQLite Database

 Meet sqlite-zstd, a Rust library that compresses your database many fold, leading to great savings in size while conserving its search capabilities intact.


As pointed out in "In Praise Of SQLite", SQLite is not a toy database:


Despite its compact size and absence of the client server model, SQLite is a RDBMS with all the features that make something relational - that is tables, indexes, constraints, triggers, transactions and the like. However, there are few built-in functions compared to PostgreSQL or Oracle. 


SQLite doesn’t have any compression features. This has changed with the Rust-based library, sqlite-zstd which promises to:


provide transparent dictionary-based row-level compression that allows compressing a sqlite database's entries almost as well as if you were compressing the whole DB file, while at the same time retaining random access.


Suffice it to say that we are all aware of the benefits of compressing data, be it that of a PDF document, a humble ZIP file or in this case a database. As a proof of concept and walkthrough of the tool, I'll work with a sample database used by the Joplin note taking app. Well work with Windows because it offers a more straightforward experience.


full article on IProgrammer:

https://www.i-programmer.info/programming/database/15686-use-rust-to-reduce-the-size-of-your-sqlite-database-.html


Labels:

Comments

Post a Comment

Popular posts from this blog

The PaSh Project gives your POSIX script superpowers by utilizing parallelization in order to speed up execution times.

The PaSh Project gives your POSIX script superpowers by utilizing parallelization in order to speed up execution times. This leads to faster results for data scientists,  engineers,  biologists,  economists,  administrators,  and programmers. I remember the time when the saying was "Learn Perl so you don't have to learn the Shell and its hundreds of utilities". Fast forward some decades and the use of shell scripts still has not been eradicated. On the contrary, their use has increased due to the rise of containers, VM's, administering the cloud, and Linux itself. This also serves as a lesson to those who are quick to denounce technologies as 'dead'. There's a time where a new use case revitalizes an old technology. So what is meant by "Unix philosophy"?  full article on i-programmer: https://www.i-programmer.info/news/90-tools/14990-the-pash-project-advancing-the-unix-philosophy-one-step-further.html    
Post a Comment
Read more

Open Source Is Not Just About Software

  It's about infrastructure as well, something that although not attracting the limelight, is as important as the open source software it hosts. Today the stewards of the largest open source registries in the world have issued an open letter demanding urgent reform in how open source infrastructure is funded, maintained, and operated.  https://www.i-programmer.info/news/136-open-source/18334-open-source-is-not-just-about-software.html
Post a Comment
Read more

DeepSWE - Best Benchmark For Evaluating AI Coding Agents?

  DeepSWE is a high-fidelity benchmarking platform designed to evaluate the performance of frontier AI coding agents on complex long-horizon software engineering tasks. Unlike traditional benchmarks that reuse existing GitHub data, this system utilizes original, contamination-free tasks across five major programming languages to prevent models from relying on memorized solutions, creating a much clearer separation between frontier coding models than older public benchmarks like SWE-bench. https://www.i-programmer.info/news/105-artificial-intelligence/19016-deepswe-best-benchmark-for-evaluating-ai-coding-agents.html
Post a Comment
Read more