Skip to main content

Use Rust To Reduce The Size Of Your SQLite Database

 Meet sqlite-zstd, a Rust library that compresses your database many fold, leading to great savings in size while conserving its search capabilities intact.


As pointed out in "In Praise Of SQLite", SQLite is not a toy database:


Despite its compact size and absence of the client server model, SQLite is a RDBMS with all the features that make something relational - that is tables, indexes, constraints, triggers, transactions and the like. However, there are few built-in functions compared to PostgreSQL or Oracle. 


SQLite doesn’t have any compression features. This has changed with the Rust-based library, sqlite-zstd which promises to:


provide transparent dictionary-based row-level compression that allows compressing a sqlite database's entries almost as well as if you were compressing the whole DB file, while at the same time retaining random access.


Suffice it to say that we are all aware of the benefits of compressing data, be it that of a PDF document, a humble ZIP file or in this case a database. As a proof of concept and walkthrough of the tool, I'll work with a sample database used by the Joplin note taking app. Well work with Windows because it offers a more straightforward experience.


full article on IProgrammer:

https://www.i-programmer.info/programming/database/15686-use-rust-to-reduce-the-size-of-your-sqlite-database-.html


Labels:

Comments

Post a Comment

Popular posts from this blog

Open Source Is Not Just About Software

  It's about infrastructure as well, something that although not attracting the limelight, is as important as the open source software it hosts. Today the stewards of the largest open source registries in the world have issued an open letter demanding urgent reform in how open source infrastructure is funded, maintained, and operated.  https://www.i-programmer.info/news/136-open-source/18334-open-source-is-not-just-about-software.html
Post a Comment
Read more

Europe Gets Its Own LLM

  EuroLLM is a fully open-sourced large language model made in Europe and built to support all twenty-four official EU languages. While several European states have separately produced their own LLMs, such as Greece's "Meltemi" or the recent offering from Switzerland "Apertus", there was no solution that catered for the languages of all 24 states belonging to the EU block. The time has come for this status to change with the appearance of EuroLLM, a foundational model that supports them all. https://www.i-programmer.info/news/105-artificial-intelligence/18448-europe-gets-its-own-llm.html
Post a Comment
Read more

The PaSh Project gives your POSIX script superpowers by utilizing parallelization in order to speed up execution times.

The PaSh Project gives your POSIX script superpowers by utilizing parallelization in order to speed up execution times. This leads to faster results for data scientists,  engineers,  biologists,  economists,  administrators,  and programmers. I remember the time when the saying was "Learn Perl so you don't have to learn the Shell and its hundreds of utilities". Fast forward some decades and the use of shell scripts still has not been eradicated. On the contrary, their use has increased due to the rise of containers, VM's, administering the cloud, and Linux itself. This also serves as a lesson to those who are quick to denounce technologies as 'dead'. There's a time where a new use case revitalizes an old technology. So what is meant by "Unix philosophy"?  full article on i-programmer: https://www.i-programmer.info/news/90-tools/14990-the-pash-project-advancing-the-unix-philosophy-one-step-further.html    
Post a Comment
Read more