Skip to main content

Posts

Showing posts with the label optimization

Database Sharding and partitioning: How to scale a database ?

Sharding and partitioning are techniques to divide and scale large databases. Sharding distributes data across multiple servers, while partitioning splits tables within one server. Sharding and Partitioning is a concept which is tightly coupled with database.   Horizontal Partitioning or Database Sharding : Horizontal partitioning, also known as database sharding. Database sharding is the process of storing a large database across multiple machines. A single machine, or database server, can store and process only a limited amount of data.  Database sharding overcomes this limitation by splitting data into smaller chunks, called shards , and storing them across several database servers. All database servers usually have the same underlying technologies, and they work together to store and process large volumes of data.  For example, suppose we store the contact info for customers. In that case, we can keep the contact info starting with name A-H on one partition/shard...

Django Optimization Processes for Write Operation for Postgresql

When optimizing a Django project for large write operations, especially when dealing with PostgreSQL, there are several strategies you can employ to reduce the time it takes to perform these operations: 1. Bulk Inserts In django, we create objects using create()  . Asynchronous version is acreate() .It's a  convenience method for creating an object and saving it all in one step.  and  These are same and equivalent. The create() method is used to create and save a single object in the database. Example: Instead of inserting one row at a time, consider using Django's bulk_create() method to insert multiple rows in a single query. This reduces the overhead of multiple database round trips. Example:  The bulk_create() method is used to create and save multiple objects in the database in a single query. It accepts a list of model instances and inserts them into the database in a single batch operation, which significantly reduces the overhead compared to individ...