Skip to main content

Posts

Showing posts from March, 2024

Python OOP

Python is an object oriented programming language. Almost everything in Python is an object, with its properties and methods. Class: To create a class, use the keyword class : class MyClass:       x = 5 The __init__() Function All classes have a function called __init__(), which is always executed when the class is being initiated.  Use the __init__() function to assign values to object properties, or other operations that are necessary to do when the object is being created Python Inheritance: Inheritance allows us to define a class that inherits all the methods and properties from another class. Parent class is the class being inherited from, also called base class. Child class is the class that inherits from another class, also called derived class. Python Polymorphism: The word "polymorphism" means "many forms", and in programming it refers to methods/functions/operators with the same name that can be executed on many objects or classes. Types of Poly...

Web Scraping vs Web Crawling

Web Scraping  Is about extracting the data from one or more web sites Web Crawling  Is abut finding or discovering  URLs or links on the web Usually data extraction project we need to use both combined crawling and scraping. For Example: We might crawl or discover urls , download the HTML files and then scrape the data from those html files. Which means we are extracting data and we know we do something with it like store it in a database or further process it in.  In web scraping we want to extract from the websites. In scraping we usually know the target websites already.  In crawling we probably don't know the specific URLs and we probably don't know the domains either and this is the reason we crawl and we want to find the URLs.  Tools:  Beautiful Soup : Beautiful Soup is a library that makes it easy to scrape information from web pages. It sits atop an HTML or XML parser, providing Pythonic idioms for iterating, searching, and modifying th...

Sum of numbers 1 to N

 Problem: We need the sum of the number 1 to N. The N could be anything (100,200,500,33,21) like anything. What is the best approach to solve this little problem ?  so there are multiple way to solve this problem.  We will discuss 3 solution .  1. Recursive way  : Using recursion we can solve this problem. Here is the solution The time complexity of this code is O(n) because it makes n recursive calls, each of which takes constant time.  The space complexity is also O(n) because each recursive call adds a level to the call stack. The maximum depth of the recursion is n, so the maximum amount of space on the call stack is proportional to n. 2. Using loop Here we need 2 different variables in our code -- a variable where we can store the sum as we iterate through the values and add them (my_sum in my code), and another variable (i in my code) to iterate over the numbers from 0 to n. The time complexity of this code is O(n). The while loop runs n+1 times, and ...

Python Closure

Imagine you have a function called outer_function that defines another function called inner_function inside it. Now, inner_function can access variables from the outer_function . When inner_function is returned from outer_function , it still remembers those variables even though outer_function has finished executing. This is the essence of a closure in Python. Example Code: In this example, outer_function takes a parameter x and defines inner_function inside it. inner_function can access x from the outer_function . When outer_function(10) is called, it returns inner_function , effectively creating a closure. Now, closure_example holds a reference to inner_function along with the value x (which is 10 in this case). When closure_example(5) is called, it adds 5 to the value of x (which is 10), resulting in 15. So, a closure "closes over" the environment in which it was defined, allowing it to access variables from its enclosing scope even after that scope has fi...

Database: How To Use GROUP BY and ORDER BY in SQL

Structured Query Language (SQL) databases can store and manage a lot of data across numerous tables. With large data sets, it’s important to understand how to sort data, especially for analyzing result sets or organizing data for reports or external communications. Two common statements in SQL that help with sorting your data are GROUP BY and ORDER BY . GROUP BY: GROUP BY is most commonly  used with SQL aggregate functions to compute statistics (such as a count of certain values, sum, average, and the minimum/maximum value in a set) for a group of rows.  Example 1 : Grouping by a single column and performing a COUNT: Suppose we have a table named "orders" with columns "product_id" and "quantity_sold". We want to count the number of orders for each product. Output: This result shows the count of orders for each product. Example 2 : Using aggregate functions with GROUP BY: Suppose we have a table named "employees" with columns "department...

Database Sharding and partitioning: How to scale a database ?

Sharding and partitioning are techniques to divide and scale large databases. Sharding distributes data across multiple servers, while partitioning splits tables within one server. Sharding and Partitioning is a concept which is tightly coupled with database.   Horizontal Partitioning or Database Sharding : Horizontal partitioning, also known as database sharding. Database sharding is the process of storing a large database across multiple machines. A single machine, or database server, can store and process only a limited amount of data.  Database sharding overcomes this limitation by splitting data into smaller chunks, called shards , and storing them across several database servers. All database servers usually have the same underlying technologies, and they work together to store and process large volumes of data.  For example, suppose we store the contact info for customers. In that case, we can keep the contact info starting with name A-H on one partition/shard...