Nowadays many of applications are data-intensive, as opposed to computed-intensive. CPU power are rarely a bottleneck of these applications, where the bigger challenge are usually the amount of data, the complexity of data and the speed which it is changing.

Table Of Contents

In this article, it will walk you through 3 important pillars when designing a robust software system — Reliability, Scalability and Maintainability, with questions and bullet-points to refresh your memory of best practises in a step-by-step manner.

What are the commonly needed functionality of data-intensive applications?

[Reliability] How do you ensure that the data remains correct and complete, even when…

Asynchronous background jobs can often dramatically improve the scalability of web applications by moving time-consuming, resource-intensive tasks to the background. These tasks are often prone to failures, and retrying mechanisms often make it even more expensive to operate applications with such jobs. Having a background queue helps the web servers handle incoming web requests promptly, and reduces the likelihood of performance issues that occur when requests become backlogged.

At Airbnb, we built a job scheduling system called Dynein for very critical use cases. Since its introduction, the system has become a very important component of our architecture, powering use cases…

What are Gorountines and Threads?

// Evaluation happens in the current go routine
// Execution happens in a new go routine
go f(x, y, z)

A goroutine is a lightweight thread managed by the Go runtime, and a thread is the smallest unit of processing that can be performed in operating systems. Besides, a thread exists within a process, it means that a single process may contain multiple threads at the same time.

For example, a web server is designed to handle a bunch of independent requests at once, so threads can be created, or taken from a thread pool in order to achieve concurrency.

Microservices are completely disrupting the way we build applications nowadays. The term has sprung up over last few years and has been increasingly being adopted by and discussed among developers. The real questions lie with what they are, how they work, etc, and hence I am putting my thoughts out there.

Table Of Contents

Part 1 gives you a heads up on microservices architecture and the technologies behind them. Part 2 focuses more on the implementation.

What are Microservices

Implementing microservices means building small, independent services…

Deciding what to study in college, l first set my mind to computer science as I had minimum understanding of its related topics.

Life as a computer science student

In retrospect, I would have never imagined myself as an engineer five years ago, building stuff from scratch. Knowledge acquired from school is scattered, from fundamental concepts of operating systems, to networking, to algorithms, etc. However, questions like “Why should l learn linux kernel”, “What is the purpose of studying networking? ”, “How can l apply algorithms in our daily lives?” has often been bubbling up in my head. …

One of fundamental step when designing an application is how you represent your data. The structure of your database’s collections and documents is one of core factors affecting your application’s speed and latency. A poor designed data representation can slow performance down by forcing you to make unnecessary queries to get the data you need.

What is Normalization?

Normalization means break down a large piece of data into small pieces with references between those collections. Each piece of data will be in a collection, but multiple documents will reference it.

What are purposes of Normalization?

It reduces the data redundancy…

Marcus Man

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store