Skip to main content

Posts

Showing posts with the label database

Database: How To Use GROUP BY and ORDER BY in SQL

Structured Query Language (SQL) databases can store and manage a lot of data across numerous tables. With large data sets, it’s important to understand how to sort data, especially for analyzing result sets or organizing data for reports or external communications. Two common statements in SQL that help with sorting your data are GROUP BY and ORDER BY . GROUP BY: GROUP BY is most commonly  used with SQL aggregate functions to compute statistics (such as a count of certain values, sum, average, and the minimum/maximum value in a set) for a group of rows.  Example 1 : Grouping by a single column and performing a COUNT: Suppose we have a table named "orders" with columns "product_id" and "quantity_sold". We want to count the number of orders for each product. Output: This result shows the count of orders for each product. Example 2 : Using aggregate functions with GROUP BY: Suppose we have a table named "employees" with columns "department...

Database Sharding and partitioning: How to scale a database ?

Sharding and partitioning are techniques to divide and scale large databases. Sharding distributes data across multiple servers, while partitioning splits tables within one server. Sharding and Partitioning is a concept which is tightly coupled with database.   Horizontal Partitioning or Database Sharding : Horizontal partitioning, also known as database sharding. Database sharding is the process of storing a large database across multiple machines. A single machine, or database server, can store and process only a limited amount of data.  Database sharding overcomes this limitation by splitting data into smaller chunks, called shards , and storing them across several database servers. All database servers usually have the same underlying technologies, and they work together to store and process large volumes of data.  For example, suppose we store the contact info for customers. In that case, we can keep the contact info starting with name A-H on one partition/shard...

Django Optimization Processes for Write Operation for Postgresql

When optimizing a Django project for large write operations, especially when dealing with PostgreSQL, there are several strategies you can employ to reduce the time it takes to perform these operations: 1. Bulk Inserts In django, we create objects using create()  . Asynchronous version is acreate() .It's a  convenience method for creating an object and saving it all in one step.  and  These are same and equivalent. The create() method is used to create and save a single object in the database. Example: Instead of inserting one row at a time, consider using Django's bulk_create() method to insert multiple rows in a single query. This reduces the overhead of multiple database round trips. Example:  The bulk_create() method is used to create and save multiple objects in the database in a single query. It accepts a list of model instances and inserts them into the database in a single batch operation, which significantly reduces the overhead compared to individ...

Database Indexing in Django application

  Database Indexing Database indexing is a technique used to optimize the performance of database queries by allowing the database management system (DBMS) to quickly locate and retrieve specific rows of data. Indexes are data structures that provide a faster way to look up records based on the values stored in one or more columns of a table. When you create an index on a table, the DBMS creates a separate data structure that maps the values in the indexed columns to the corresponding rows in the table. Default Type of Index is B-Tree Index ( The king of all indexes) বইতে কোন টপিক খুজতে গেলে আমরা টেবিল অফ কনটেন্ট থেকে দেখি এই টপিক কত নম্বর পেজে আছে।যাতে করে আমাদের পুরো বই খুজতে না হয়। ডেটাবেজ ইনডেক্সিং ও তেমনই একটা ইফিসিয়েন্ট টেকনিক।ডেটাবেজে কোন ডেটাকে দ্রুত খুজে বের করার জন্য ইনডেক্সিং করা লাগে।যদি এমন হয় একটা কুয়েরি বার বার এক্সিকিউট করতে হচ্ছে এবং একটা কলাম থেকে ভ্যালু বার বার খুজতে হচ্ছে তখন আমরা সেই কলামে ইনডেক্সিং করতে পারি।এর মাধ্যমে কোন ডেটা দ্রুত রিট্রাইভ করা যায়।কিন্তু ই...