Redis is popularly known for being used as a cache store alongside Relational Database Management System (RDBMS). As it is an in-memory data store, the read/write speed is higher than traditional RDBMS, which stores data on the disk. However, there are downsides to this database as well. For instance, there are memory limitations as all the data is stored in memory (or RAM), and a large amount of RAM can significantly increase the cost of the server on the cloud.

Using Redis as a Cache

Image credit: backendless.com

Redis can be viewed as an intermediate data store or cache that temporarily stores frequently requested data from the database so that database operations are reduced. When the data is not present in the cache, it is retrieved from the persistent database and cached to Redis for further retrieval.

Redis as a Primary database

When we think of using Redis as a primary database, we need to figure out how to store all of our application’s data on Redis, i.e., user data, login credentials, relationships, indexes, etc. Redis has grown from simply storing key-value pairs to storing multiple data structures like Sets, Bitmaps, Hyperloglogs, and others.

Database design of Shortomega

As key-value stores like Redis don’t have rows, columns, tables, or relationships, I need to devise a solution.

Let’s start with a simple data — user’s email and password:

Each user has a unique email and corresponding password to log in. We typically store the hashed value of the password using a hashing algorithm like SHA-1 or SHA-256 to protect the plain password from attackers.

We can store email ID with the corresponding user ID by adding the prefix “email” to the key:

email:<userid> -> <email address>

And the hashed password using:

password:<password> -> password

However, there are some issues with this approach. Whenever we need to add new attributes to the same user, we must add a new hash with a new prefix every time. This will significantly increase the memory space needed to store each user's attributes. Moreover, we usually need to set and retrieve email and password together and for that, we have to perform 2 database operations.

Redis has a built-in structure called hashes which is a collection of field-value pairs. We use the HSET command to add fields to the hash. HSET syntax looks something like this:

HSET key field value [field value ...]

Now we have only one hash for each user data and can store data in this way:

HSET user:<userid> email mail@test.com password <hashed-password> [...other fields and value]

We can easily retrieve all fields using HGETALL command:

Now that we have successfully stored user data efficiently, I will explain how I am building a URL shortener application — Shortomega using Redis as the primary database. I am storing the user login credentials similarly as explained before.

To store short URLs and their corresponding long URL, I am simply storing them in key-value pairs to query them in O(1) time.

short:<short-url> -> <long-url>

In order to associate a short URL to a user, I created a set of short URLs for each user as shown below so that I can retrieve all the created short URLs in a single Redis query.

user:<userid>:urls -> [set]

Challenges

Using sets to store short URLs created by a particular user, I need to perform additional N queries to retrieve corresponding long URLs. This is similar to the N+1 query problem in databases. To address this, I needed to choose between these 2 options:

1. Store the long URL along with the short URL for a particular user. This will result in redundancy. Managing and updating the URLs at the two places would be cumbersome.

2. Another option is to write a custom Lua script that runs on the Redis instance and returns the pairs of short and corresponding long URLs. This solves the N+1 query problem and gives the result in a single query. I chose to use this method. The script looks something like this:

local urls = redis.call('SMEMBERS', KEYS[1])
local result = {}
for i, url in ipairs(urls) do
    local longUrl = redis.call('GET', 'short:'..url)
    if longUrl then
        table.insert(result, url)
        table.insert(result, longUrl)
    end
end
return result

In the above Lua script, I am fetching all the short URLs based on the user ID passed from the NestJS backend. By looping over each URL, I get the corresponding long URL that is simply stored in a key-value pair. Lastly, I insert the pair in a table which I return to the back-end for further processing.

To benchmark the performance of retrieving the data in a single database query, I ran a Python script where I first stored and retrieved the data using N+1 queries and I executed Lua script on the Redis instance to retrieve the data in one single query. I plotted the graph as shown below:

In the above plot diagram, we can see the execution time difference between N+1 queries and the Lua script. A user with around 5000 URLs can fetch all the URLs in under 50 ms instead of 300 ms.

Analytics

Analytics include counting of total and unique visitors for each short URLs. This helps the user to track the engagements of created short links across different regions.

The count of total visits can easily be stored in a key-value pair. The value is incremented each time someone clicks on the link. But what about unique visitors? I can store unique IP addresses of visitors in a set and then count the elements in the set to get the unique visitors. However, it will take O(N*M) space in the database to store all the N IP addresses of M links, consuming the memory of the server. As mentioned before, increasing the RAM of a server instance on the cloud is more expensive than adding more disk space. It turns out that there is one special built-in data structure in Redis that can solve this problem. I am talking about HyperLogLog.

HyperLogLog is a probabilistic data structure that estimates the cardinality of a set. Basically, it estimates the number of unique elements in a set without the need for actual storage of elements in a set. It uses up to 12 KB of memory irrespective of count of elements in the set and provides a standard error of 0.81%.

We can add and count elements using following syntax:

PFADD key [element [element ...]]
PFCOUNT key [key ...]

I can now add the IP addresses to the HyperLogLog and count the unique IP addresses.

PFADD short:<hash>:ips [IP addresses ...]
PFCOUNT short:<hash>:ips

Lastly

There are endless possibilities of how we can use Redis as a cache and even as a primary database. The fast operations and in-memory operations of Redis make it an ideal choice for high-performance and real-time applications.

The URL shortener application — Shortomega is work in progress and I will add further development and details here or in another blog.

Enjoy Building!

The internet started in the 1960s as a way for government researchers to share information. It was not until 1991 when computer scientist Tim Berners-Lee invented the World Wide Web (WWW). He created the browser to access the web and made publicly available. That's when internet reached to the common people all around the world. At first, people could only send emails to communicate with other people on the internet. This version of web was named as Web 1.0. After the rise of social networking sites like SixDegrees, MySpace, Facebook, Twitter, Instagram and Snapchat. Then new version was made named Web 2.0.

You affect the world by what you browse.

— Tim Berners-Lee

Since every coin has two sides, the advantages of social media are worth mentioning. It is easy to find new people of same interest, talent or profession. On social media sites like Facebook, we can find the groups of our interest, follow influencers and make new friends from anywhere in the world. Sharing ideas, creating new things, talking about serious topics, spreading awareness around the globe and many more things have been made possible by the community on the internet. Some creators crowdfund for good cause and needy ones managed to get organs with the help of the social media. During the global pandemic, many people seek out hobbies, interest, attend live concert online, connect with friends on video call and educate themselves with genuine news about the pandemic.

Internet is a volatile place. One can never predict what would possibly happen in very short span of time. With over 4.5 billion people on the internet, a large number information is shared on the web which can be accessed by everyone on the web. As the consequence, there's a lot of content uploaded every single second. If not regulated properly, anyone can read the content which leads to misuse of the shared information. People might get exposed to age-restricted content, fake news, conspiracies, illegal digital materials, and many more. Speaking of fake news, Covid-19 fake news spread faster than virus itself. There was violence, protests in many countries. In fact, people didn't cooperate with local authority which made even more difficult to curb the virus. As a result, social media can be useful at a certain extent but it can surely overwhelm us.

Social media addiction is another major problem especially among teenagers. According to Addiction Center, social media addiction looks much like any other substance disorder and may include mood modification, behavioural and emotional changes, ever-increasing use of social media, experiencing unpleasant physical and emotional symptoms when social media use is restricted and conflict. These trillion dollars-per year companies produce the same sense as of gambling and drugs to keep users using their products as much as possible. We have seen people considering the amount of likes and comments on their posts and worrying about it, constantly checking their social media apps. We tend to make a lot friends online but the question is whether they are our real friends. We chat a lot on Instagram, Messenger, and WhatsApp but definitely don't know the mood, expression, emotion and attention of the person with whom we are texting. Talking in front of that person makes the conversation real because we can see their body gestures, look, voice, etc. Teenagers are facing more depression, anxiety and loneliness than ever.

Likes just changed from spreading love around the world to making teenagers depressed.

Big tech corporation are very good at making their products as addictive as possible. These companies collect a huge amount of data even if you are not using their products. Facebook and Instagram collect your browsing history, posts you like, people you follow, your exact location, news you read and undoubtedly sell these data to advertising companies to show you ads. It seems fine as these platforms run on advertising business models but it's hard to imagine the amount of data collected and processed to target us with relevant ads. Not only that, intruders can attack these platforms and steal users' data. In Facebook-Cambridge Analytica scandal, Facebook confirmed that data of potentially 87 million users was collected without the users' consent for political advertising. The data contained users' profile photo, birthday, current city, page likes, news feed, timeline, messages and 5000 different data points. The data was detailed enough to create psychographic profiles of the users. As mentioned in popular documentary The Great Hack, Cambridge Analytica is just the tip of the iceberg.

If you are not paying for the product, you are the product.

Internet was created to decentralize the mode of communication which means there is web of network and even if anyone network collapses, the internet does not get disrupt. Contrary to this, internet is mostly centralized by tech companies like Facebook and Google today. Most of the people feel internet is Facebook and Google and they are not wrong. During the recent Facebook, Instagram and WhatsApp outage, many people suffer because businesses work on it. People could not log into other web services with Facebook. Therefore, it is clear that Facebook has created monopoly throughout the internet.

Data is the most valuable asset on the Earth.

Now it is our responsibility to control the use of social media. No one would suggest to delete social media accounts to everyone. But even if few people do, it matters a lot. Here are the suggestions for everyone:

1. Turn off all the notifications. Don't let unnecessary stuff disturb your peaceful mind.

2. Turn off any personalised ads option.

3. Check multiple times before posting anything online. Even if you delete it later, it still remains in the system.

4. Don't spread fake news. Fact-check it always on Google or any other trusted source.

5. Set screen time limits on your phone.

6. Avoid keeping your phone in the bedroom at night.

7. Don't watch what YouTube recommends you, always search for the video.

8. Keep social media apps out of your sight which means out of your home screen.

9. If you feel uncomfortable, talk about it with seniors.

10. Take your time. Do something exciting. Read a book, go for a walk.

11. Make friends in your neighbourhood.

12. Lastly, get out of these AI-generated world. Look outside. The world is beautiful.

Whether it is to be Utopia or Oblivion will be a touch-and-go relay race right up to the final moment....

— Buckminster Fuller

We get Windows 10 preinstalled with most of the computers and laptops. macOS is installed in Mac lineup manufactured by Apple. These two operating systems are proprietary which means no one except the owning companies can access the code and contribute in the development of them. On the other hand, Linux is an open source operating system. It is owned by a community of developers. These people develop it and fix any errors or bugs in the OS. It is free of cost and can be used on any hardware. Linux has many distributions like Kali Linux, Ubuntu, Fedora, Arch Linux, Peppermint OS, Linux Mint, Elementary OS, etc. You can either buy a PC with Linux preinstalled or use Linux along with Windows 11 using Dual-Boot or installing it on a Virtual Machine like VMWare Workstation or VirtualBox. Here we are going to see how to install and setup Ubuntu using Dual-Boot because it is the most popular and easy for beginners.

Pros

1. It is less susceptible to viruses and malware.

2. It is easy to install.

3. It does not take much disk space and can be run on any hardware system.

4. It has powerful command prompts.

5. It does not crash or requires reboot frequently like other OS.

Cons

1. The user interface is not as good-looking as the other OS.

2. It requires practice and skills.

3. There is limited support for popular software and apps.

4. Some hardware drivers are not available for Linux.

5. Not useful for gamers as games are popularly made for Windows.

As we can clearly see Linux OS is not useful for everyone. If you are a gamer then Windows is best for you. If you are a productive user then Linux has a lot of support and features to help you.

Basic Installation

Prerequisites: A PC with Windows installed and minimum 30 GB disk space, USB pen drive (minimum 8 GB), Active internet connection.

• Step 1: Create a backup of your PC [Optional]

  It is a good practice to frequently make backups of your computer in a separate external disk. I would recommend you to take backup before installing Ubuntu in case you get stuck into a major boot issue.

• Step 2: Download Ubuntu ISO file

  You can download Ubuntu ISO file from the official Ubuntu website . Make sure you download latest version available. The latest version is Ubuntu 20.04 (Focal Fossa).

• Step 3: Create live USB stick

  To create a live USB stick, download and install Universal USB Installer . After installing, connect your Pen Drive and open File Explorer and format it. Open Universal USB Installer and browse the location of Ubuntu ISO file. Leave everything default and click start. It will automatically create a live USB of Ubuntu.

• Step 4: Boot into Ubuntu

  This is the most important step. Connect the pen drive to the computer you wish to install Ubuntu. Click start button and type Change advanced startup options and press Enter. Under the Advanced startup, click Restart Now. Windows Safe Mode page will show up. Choose Use a device option and click the USB disk. It may also be displayed as EFI USB Device. Clicking it will automatically boot into Ubuntu OS.

• Step 5: Installing Ubuntu

  

  When the Ubuntu boots up, you will see two options. You can either try Ubuntu or install it. We are going to install it on the PC so select the second option.

  

  After that choose your language and keyboard layout.

  

  On the next screen choose Normal installation and tick Install Third-Party Software and click continue.

  

  On the next page choose install Ubuntu alongside Windows.

  

  After that allocate the disk space you want to give to Ubuntu and click Install Now. It will take 8-10 minute to install. Once the installation is complete restart your PC. Remove the USB and press Enter.

Everything is done. Now just set up your home screen as per your need. Now it is important to check for updates so that everything runs smooth.

Installing Updates

To install updates, go to app drawer and open Software Updater. It will automatically start updating. Make sure you sign up to Canonical Livepatch. It will frequently install security patches.

Fixing Issues

Even if you do everything right, there are some issues which will appear. Let's see some common issues and fixes.

1. Wi-Fi issue: This is the most common issue which everyone talks about on the internet. It's pretty easy to fix this. This happens because of missing Wi-Fi driver. To fix this, update the software from Software Updater and then open Additional Drivers. There you will see the Wi-Fi driver available for Ubuntu. Tick use option and click Apply changes.

   

2. Bluetooth issue: I don't know how many people face this issue but I got this on my PC. I search for fixing this but haven't found the solution. I will update if I find anything.

3. Display flickering issue: Even this issue is prominent on the internet and I still haven't found any fix for this. It may have happened due to incompatible Graphics Driver. I will update once I find anything.

Accessing Windows Files

Switching OS doesn't mean you can't access another OS files. You can actually access your photos, videos, documents, audio files present on Windows. To do this, go to Files and click Other locations. You will see all your drives (C, D, E,…). Just click on the desired drive and you can read the files on it.

Command Lines

Terminal of Linux is far more powerful than Windows Command Prompt. You can do a lot of things with just typing commands. We will see some of the common commands.

cd [folder name or path]: This lets you to change the directory.

ls: This simply displays all folders and files in the directory.

mv: Move a file

cp: Copy a file

sudo: It gives root access to any command. You must enter your password to use this command. It requires you to type before every command. Use sudo -s to avoid typing sudo for every command.

Downloading apps

Downloading apps is easier on Linux than any other OS. You can either head to Ubuntu Software Store and install your favorite apps or just type command in the terminal. We will see the second method.

Installing apps using apt

sudo apt install [package-name]

Installing apps using Snap Store

sudo apt update
sudo apt install snapd
sudo snap install core
sudo snap install [package-name]

Commands for popular apps

1. Spotify

    sudo snap install spotify
   

2. VLC media player

    sudo apt install vlc
   

3. Audacity

    sudo snap install audacity
   

4. Android Studio

    sudo snap install android-studio --classic
   

5. Sublime Text

    sudo snap install sublime-text
   

Using GNOME Shell Extensions

Extensions are pretty useful and enhances the overall experience. With GNOME, we can have a bunch of useful extensions from keeping all copied text to changing desktop wallpapers daily. There are some things to install before using extensions.

First check your GNOME shell version. It should be 3.36 or above

gnome-shell --version

Add Extension Integration on browser

For Chrome | For Firefox

Install Native connector

sudo apt install chrome-gnome-shell

Install GNOME tweaks

sudo apt install gnome-tweaks

That's it. Now you can go to extensions.gnome.org . There you can browse and enable extensions. To enable click on it and toggle the on or off switch. Then enter your password.

Here are my favorite GNOME extensions:

• Arc menu

• Clipboard Indicator

• GSConnect

• NetSpeed

• Transparent Top Panel

• Caffeine

• Google Earth Wallpaper

Well that's it for this guide. I hope it helped you to successfully install Ubuntu and get started. There is a lot to learn here. You can search more about this on Google and YouTube. If you have any questions or doubts, you can email me. Thanks a lot.

Here I am going to explain what is this new type of currency in simple words. I want to address that I do not know everything about cryptocurrency so it is difficult for me to discuss everything here. So, let's start from the basics.

What is Cryptocurrency?

Cryptocurrency is a digital money. It is not physical money. It is all online. That's why you won't see a person giving crypto coins to the cashier. You can transfer cryptocurrency to someone else online without a middle factor like a bank. Bitcoin is the most popular cryptocurrency.

Cryptocurrencies work on the blockchain technology. Blockchain is a secure decentralized technology spread across many computers all over the world. These computers manage transaction records.

As cryptocurrencies are decentralized, there is no centralising organisation or any Government body which manages the market value of a particular cryptocurrency. This is one of the many reasons why the cryptocurrencies are super sensitive. This means that their value increases or decreases unprecedentedly.

How does cryptocurrency work?

Cryptocurrency is mined (in other words earned) using powerful supercomputer. This supercomputer is given a complicated mathematical puzzle to solve. If the computer solves this puzzle, then it is rewarded with some cryptocurrency. This puzzle is nothing but the records of previous crypto-transactions which is placed to the next block in the blockchain.

Is this the only way one can earn cryptocurrency? No. Cryptocurrency can be bought using real money. Think it as the Casino Coins. It is exchanged with general currency. This makes it an ideal asset for investors because its market value changes all the time like a share value or gold value.

What is a Blockchain and how does it work?

Blockchain is a chain of blocks which contains information. This helps for the sequential arrangement of data which is very difficult to change it. Each block contains some data along with hash of the block and hash of previous block. The data stored depends on the type of the blockchain. A bitcoin blockchain contains details of the transactions such as the sender, receiver and amount.

A block includes hash. The hash is alphanumerical data and it is unique like fingerprint. Any changes made in the block causes the hash to change. That's why hash is very important to verify the data in the block. The interesting part of the blockchain starts here. If someone tampers with the data in a block, the hash will change but the next block does not contain the new hash of previous block. This makes the following blocks invalid. To check the changes made in the block is authentic or not, all the system having access to it need to verify. This makes the blockchain technology so secure. Learn more about this in the YouTube video here.

Is cryptocurrency legal?

Cryptocurrency is not illegal. Anyone can sell, buy and trade cryptocurrency. In the US, cryptocurrencies are defined as a property rather than the currency. In India, the mining and trading of cryptocurrency is not legal. The Government does not consider cryptocurrency as legal tender or coin. The right intention is to prevent the wrong usage of cryptocurrency and blockchain. The Government will make use of blockchain technology in few years.

Advantages

As the cryptocurrency is virtual money, there are some key advantages over traditional currency.

1. Transaction: In the traditional businesses, there is brokerage, commission, agents, paperwork which makes the process complicated. Cryptocurrencies are traded without the need of ‘middle man'. This is known as one-to-one affair.

2. More confidentiality: The transaction details are not publicly available unless you do so. This protects your financial history.

3. Less transaction fees: As there is no bank system, you do not require to pay transaction fees. You are maybe required to pay to crypto wallets.

4. Greater access: These services are accessible to anyone having internet. It is estimated that people have more access to internet and mobile phones than banking services like credit cards.

5. Strong security: We have seen that blockchain technology is highly encrypted. So, it is impossible to hack this system.

Disadvantages

This system has some major disadvantages which prevents it to become wide-spread.

1. Illegal transaction: This problem is concerning many countries. As the transaction details are private, the investigating organisations cannot access these details. So many illegal items like drugs are bought on the dark web. Bitcoin is the biggest culprit here.

2. Financial losses: If a user loses its wallet key and password, there's no getting it back. The wallet will be locked along with the number of coins in it.

3. Adverse effects on environment: Mining cryptocurrency requires advanced computing power which cannot be done using ordinary computers. This demands a lot of energy.

4. Susceptible to hacks: Although cryptocurrencies are very secure, the wallet data is not that secure. Hackers can access this data and a lot of money can be stolen.

5. No refund: If someone mistakenly sends money to wrong address, there is no way to get it back. Many people can cheat others and create transaction whose products or services the user never receives.

Conclusion

Cryptocurrencies have a long way to go. They are just at the initial stage. Many individuals and organisation are creating their own cryptocurrency. Even you can create your own cryptocurrency. This digital currency has become a good investment for investors. Cryptocurrencies might be banned in several countries but blockchain technology is more acceptable around the world. Indian Government is considering to release their own cryptocurrency using this technology.

As the result, if you have little knowledge about cryptocurrency, you can get started with it. Hope it helped you to understand. Please share this blog with your friends. Thank you!

PS: This blog was published on my personal blog page on April 29, 2021